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CHAPTER I 



THE SIGNIFICANCE AND AMBIGUITY OP 
INFORMATION ON TEACHING 

Ths puiposs of 'bhls study Is to delinsats ths conditions undor 
which statewide programs for measuring and evaluating teaching in 
elementary and secondary schools can yield optimum information for 
use hy the states in carrying out their educational leadership responsi- 
hilities. These responsihlllties relate particularly to three state 
services administered through state departments of education: the 

education of teachers, the certification of teachers, and the 
accreditation of schools. 

To achieve its puxpose, the study describes possible approaches 
for measuring and evaluating teucV-ix^G and proposes a plan for develop- 
ing a state program for colleccxtig -‘ud analyzing information on 
teaching. A review of the significance of such an undertaking, dong 
with problems involved, is presented in the present chapter. 

The Significance of Infoim ation on Teaching 
7 for State Educational Leadership 

State educational leadership is defined here as the initiation 
of new structures or procedures for accomplishing educational objectives 
of a state or for changing these objectives . This definition is based 
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upon one proposed “by Hemphill^ and used also by Lipham.^ The latter 
author contrasts leadership and adminxstration, defining adminis- 
tration as the utilization of existing structures or procedures to 

“3 

accomplish established goals. 

State governments have a mandate to carry out both leadership 
and administrative responsibilities in education, since public educa- 
tion is a state responsibility. The Tenth Amendment of the federal 
Constitution assigns to the states those powers which are neither 
reserved to the federal government nor denied to the states. Among 
these is education. Admittedly, a great deal of discretion has tra- 
ditionally been exercised by local school corporations; this has been 
possible only by virtue of authority granted by state governments. 

Every state possesses a large measure of power in the C4xea of educa- 
tion which it has not yet begun to exercise. Also worthy of note is 
the fact that the role of the federal government in education has ex- 
panded greatly in recent years; such involvement has been possible only 
under the justification' that it promotes the general welfare. Each of 
the fifty states holds the authority to determine the conditions under 
which schools shall be established, the qualifications necessary for 

^John K. Hemphill, "Administration as Problem Solving, " Admin- 
istrative Theory in Education, ed. Andrew W. Halpin (Chicago: Midwest 

Administration Center, University of Chicago, 1958)> P* 98* 

■ ^ James M. Lipham, "Leadership and Administration," Behavioral 
Science and Educational Administration, The Sixty-third Yearbook of 
the National Society for the Study of Education, Part II, ed. Daniel 
E. Griffiths (Chicago: The University of Chicago Press, 1964), p. 122. 

^Ibid. 
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p6rsons who "boach in "bh© schools > and "bh© sp©cific subj©cts which can 

and shall h© taught in th© schools. 

Each stat© gov©mm©nt ©x©rcis©s l©ad©rship by ©stablishing con- 
stitutional provisions for an education system and by establishing 
policies intended to maJee it possible for each child within the state 
to secure a high quality education. It is generally acknowledged that 
the quality of teaching which takes place is an iajortant factor in 
determining the quality of an educational program. Hence, state leader- 
ship has resulted in curriculum guides and .consiiLtative services to 
help improve teaching, plus comprehensive regulatory systems to protect 

the public from incompetent teaching. 

These regulatory systems are comprehensive in that their effects 
are felt at three different stages : in teacher preparation, at the 

completion of teacher preparation, and throughout a teacher's tenure. 
States regulate teacher preparation by requiring that teacher education 
institutions be granted approval by a state agency.^ At the comple- 
tion of pre -service programs teachers in all states must be granted 

2 

state certificates before they are eligible to teach in public schools. 
Conditions under which certificates can be renewed or extended are pre- 
scribed by the individual states. Accreditation or other regulatory 



4his practice is followed in all states except Hawaii, New 
Mexico, and North Dakota. John R. Mayor and Willis G. Swartz, Accredi- 
tation in Teacher Education: Its Influence on Higher Education 

(vfflgTrfngtnnj D. fi. ! National Commission on Accrediting, 19 ^^)} P* 23. 

^W. Earl Armstrong and T. M. Stinnett, A Manual on Certification 
Requirements for School Personnel in the United States (Washingten, 
p, C. : National Commission on Teacher Education aM I^ofessional 

Standards, Rational Education Association, 1964). 
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systems are established to prevent teachers from being assigned in 
areas for which they lack the necessary qualifications and to eliminate 
other sub-standard teaching situations. 

However, to exercise further leadership through revision of 
the regulations and improvement of the services, information which is 
immediately relevant to the services is required. Since the services 
are directed toward improving teaching, information on the nature of 
the teaching which is taking place in the state is most relevant. With- 
out such information, leadership must be based upon less sifmificant 
evidence . 

If those persons who exercise control over the services receive 
no information relative to the effectiveness of their operation, it is 
probable that no change will be initiated (i.e., no leadership will 
take place). This idea of organizational inertia is an in 5 )ortant 
postulate in "general systems theory, a meta-theory or super- theory 
which has been suggested to provide the framework for concepts in a 
number of areas and disciplines,. The theory provides a series of 
postulates relative to the functioning of systems. A system is a com- 
plex of elements in mutual interaction. These elements may be chemicals, 
animals, people, groups of people, organizations, nations, planets, or 

virtually anything else. Griffiths, who is cited above, deals primarily 
% 

with social organizations, employing the theory as a means of investi- 
gating the problem of change in organizations. 

^‘Daniel E. Griffiths, "Administrative Theory and Change in 
Organizations, " Innovation in Education, ed. Matthew B. Miles (Ne".: York: 
Bureau of Publication, Teachers College, Columbia University, 1964 ), 
pp. 425-436. 
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Central in systems theory is the role of feedback^ information 
received from the environment vhich can be used in regulating future 
activities of the system. Through the use of feedback, systems regu- 
late themselves. If the controlling elements of a system receive no 
relevant feedback, they will have no basis for initiating change. 

To revise standards and policies pertaining to the education 
of school personnel, the certification of school personnel, and the 
accred.-*.tation of schools, information on teaching which is taking 
place and which has taken place is prime feedback. Measurements and 
evaluations of teaching could make available qualitative evidence aa 
to the effectiveness of those state services which are established to 
provide an adequate supply of qualified teachers. These Include re- 
cruitment activities, inrservice education programs, and approval of 
pre-service education programs. In teacher certification, measure- 
ments and evaluations of teaching could provide an indication of the 
validity of certain aspects of these requirements. In the accredita- 
tion of schools, information on the relationship between various 
school charsuiteristics and quality of teac hin g could be very useful 
in developing accreditation standards and procedures. 

Present Use of Information on Teaching 
for State Educational^adership 

As much significance as the potential ^ich information on 
teEichiug appears to hold for improving educational leadership decisions, 
it may seem ironic that virtually no systematic use is presently made 
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of such information by state governments. As the survey reported in 
Chapter II indicates, only three states have attempted to establish 
programs for evaluating teaching. Mayor and Swartz report no use 
of Information on teaching performance of graduates in programs for 
the approval or accreditation of teacher education.^ No evidence was 
found of the use of information on teaching in state programs for 
teacher certification.^ In the area of school accreditation, Grizzell 
reports that it is generally acknowledged that present secondary school 
standards have not been validated, but are largely adaptations of 
the standards originally adopted by the North Central Association in 

1902.^ 



The Ambiguity of Infomation on Teaching 
The principal explanation for the rare use of information on 
teaching in situations where its potential contributions appear so 
great must certainly relate to the ambiguity of that infomiation. 
Virtually any information on teaching which may be gathered is subject 
to multiple and variant inteipjretations . Three factors contributing 
to this situation are (l) differing concepts of what criteria should 
be used in evaluating teaching, (2) limited understanding of factors 
which make teaching effective or ineffective, and ( 3 ) factors other 
than teaching which affect the outcomes of teaching. 

"Hdayor and Swartz. 

^Armstrong and Stinnett. 

%. D. Grizzell, "Accreditation: Secondary Schools," Encyclo- 
pedia of Educational Research, ed. Chester W. Harris (New York: The 

Macmillan Cooqpany, I960), p. 17* 
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Criteria for Evaluating Teaching 

If information on teaching is to have any utility, there must 
he available some explicit framework for use in interpreting it. 

Without this framework, such information is mere description with 
no extrinsic value. Hence, the first step in interpreting informa- 
tion on teaching consists of selecting relevant criteria with which 
the information can be evaluated. These criteria must, of course, be 
pertinent to the purposes which the teaching is ej^ected to fulfill. 
When different evaluators enploy different criteria, their interpre- 
tations of the same information on teaching will pTObably be different. 
This fact was demonstrated by Anderson.^ Such differences are one 
source of ambiguity in information on teaching. In the past, there 
has been much disagreement as to what criteria of effective teaching 
should be adopted. 

Mitzel^ proposed three classifications for teaching effective- 
ness criteria. These can be viewed on a goal-proximity scale: (l) 

product criteria, (2) process criteria, and (3) presage criteria. 

The enployment of any one of these types of criteria calls for measures 
of teaching which are different from those required by the other types. 
Hence, the type of criterion employed dictates the type of information 
oh teaching which is relevant. Product criteria relate directly to 

^Harold M. Anderson, *’A Study of Certain Criteria of Teaching 
Effectiveness," Journal of Eaperimental Educatian, Vol. 23 (September, 
195^), PP- ^1-71. 

^Harold E. Mitzel, "Teacher Effectiveness," Encyclopedia of 
Educational Research, ed. Harris, pp. 1482-85. 
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the goals toward which teaching is directed: changes in the behavior 

of pupils. Process criteria comprise types of student and teacher, 
behavior believed to be inherently worthwhile. Presage criteria in- 
clude traits or eaqperiences of teachers which are conjectured to be 
relevant to product oi* process criteria: personality characteristics^ 

knowledge^ and swiademic achievement are illustrations. 

As convenient an it would be to have a standardized set of 

criteria of good teaching, it is unlikely that any with wide acceptance 

will be esteODlished in the near future. As indicated above, criteria 

must be pertinent to the objectives which the teaching is estpected to 

accomplish. Vflien the nature of the teaching goals varies, the criteria 

for evaluating the teaching may vary also. Such a conclusion was reached 

by the Committee on the Criteria of Teacher Effectiveness which was 

established in 1951 by the American Educational Research Association. 

The committee report states that, . .in actual research and service, 

criteria of teacher effectiveness might differ radically according to 

culture, level, method, curriculum and the like.”^ Brown thinks that 

tMs is not only true, but right. In describing the University of 

Wisconsin Teacher Conpetence Project (now the Florida Teacher Cooqpetence 

Project), he referred to the use of variable (and even conflicting) 

2 

criteria as the most acceptable approach in a democracy. To avoid 

H. Remers, et al ., ’’Report of the Conmittee cn the Criteria 
of Teacher F'^f ectiveness , ” Reviewr of Ed ucational Research, Vol. 22 
(June, 1952)^ P* S4l. 

^[Bob Burton Brown], ’’Teacher Coopetence Project” (Madls^, 
Wisconsin; School of Education, The University of Wisconsin, 19w)> 
p, 4. (Mimeogra^ed. ) 
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ambiguity when applying this concept, it is necessary to be aware of 
the explicit criteria employed by the evaluator in order to interpret 

his evaluations. 

In sum, it is necessary to employ definite criteria in inter- 
preting information on teaching. Yet, with many criteria available, 
it seems doubtful that universal acceptance for any one set can be 
attained. Hence, in any given teaching situation, it is necessary 
first to reach agreement as to the criteria which are applicable, 
only after this is it reasonable to expect agreement in interpreting 
information on teaching which has taken place in that situation. This 
same conclusion was reached by Yildirlm.^ He states that, if the 
disagreement arises in an area of value judgment (i.e., it relates to 
the criteria which should be applicable), reconciliation is contingent 
\jpon the willingness of the parties involved to reconsider their basic 
premises and to seek consensus. If, on the other hand, it relates 
to a factual judgment (i.e., it pertains to the specific content of 
information collected), the disagreement can be resolved by collecting 
additional Information or by some other means of appealing to external 
measurements or standards. 

The Status of Kaowledge About 
Effective Teaching 

Information relating to the process or prerequisites for teach- 
ing is ^0 subject to multiple interpretations because little is known 

^Cemal Yildirlm, "An Analytic Model for Evaluation of Teacher 
Coopetence" (unpublished Hi.D. dissertation, Indiana University, 
Bloomington, 1963)> PP* 69-7^. 
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about "what makes the vork of an individual teacher effective. Research 
attempting to elucidate this topic has been extensive, but the findings 
■with 'practical applicability are meager. 

The volume of studies which have been completed can be appre- 
ciated by perusing bibliographies by Domas and Tiedeman^ (1006 entries) 

O 

and Morsh and Wilder (362 entries). However, the results of these 
and other efforts were judged by Remmers to contain little information 
". . . that a superintendent of schools can safely employ in hiring a 
teacher or granting him tenure, that an agency can employ in certify- 
ing teachers, or that a teacher education faculty can employ in planning 
or iii5)roving teacher education programs."^ Turner and Fattu state that, 
"Seventy years of research on teacher effectiveness have not added much 

to our systematic knowledge, and it is difficult to see how another 

k ... 

seventy can do any more if the same procedures are followed. " U'cner 
■writers expressing dissatisfaction with research results and methodology 



^S. J. Domas and D. V. Tiedeman, "Teacher Competence: An 

Annotated Bibliography," Journal of Experimental Education, Vol. 19 
(December, 1950)^ PP» 101-21b. 

^J. E. Morsh and E. W. Wilder, Identifying the Effective In- 
structor: A Review of Quantitative Studies, 1900-19^2 (Research Bulle- 
tin No. AFITRC-TR- 55-44, San Antonio, Texas: United States Air Force 

Personnel and Training Center, 195^)- 

^H. H. Remmers, et a l., "Second Report of the Committee on 
Criteria of Teacher Effectiveness," Journal of Educational Research, 
Vol. 46 (May, 1953), p. 657. 

^Richard L. Turner and Nicholas A. Fattu, "Skill in Teaching, 
Reappraisal of the Concepts and Strategies in Teacher Effectiveness 
Research," Bulletin of the School of Education, Indiana University, 

Vol. 36 (May, i960), p. iii. 
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include Barr aiid Jones* Mitzel^ and Ryans# 

Some writers, however, esqpress optimism in view of current 
approaches to analyzing relationships between various aspects of 
teacher behavior and subsequent pupil behavior. Cogan is pessimistic 
about the past but sees hope for the future.^ He sees new instill- 
ments for data collection brought on by modem technology as a key to 
in5>roving research on teaching. Mitzel also indicates confidence that 
recent trends in educational research will ultimately lead to knowledge 
of patterns of effective teacher behavior.^ He places his hope in 
massive research efforts conducted by interdisciplinary teams with 
ample financial support. Soar feels that a new era of research on 
teaching may now be underway: **. . one in which real progress is 

being made in the specification of dimensions of classroom process 
that are effective in producing pupil change.’*^ 

^Arvil S. Barr and Robert E. Jones, "The Measurement and Pre- 
diction of Teacher Efficiency," Review of Educational Research, Vol. 28 
(June, 1958), EP» 256-264. 

Stttzel, "Teacher Effectiveness," Encyclopedia of Educational 
Research, ed. Harris, pp. l48l-l486. 

^David G. Ryans, "Theory Development and the Study of Teacher 
Behavior*” Journal of Educat ional Psychology, Vol. 4-7 (December, 1956)^ 

pp. 462-475- 

Sforris L. Cogan, "Research on the Behavior of Teachers: A 

New Phase,” Journal of Teacher Education, Vol. l4 (Septeaiber, 1963)^ 

pp. 238-243. 

^Harold E. Mitzel, "Can We Measure Good Teaching Objectively?” 
NBA Journal, Vol. 53 (January, 1964), pp. 35-36. 

%dbert S. Soar, "Observation Systems and Research on the 
Teaching-Learning Process,” Classroom Interaction Newsletter, Vol. 2 
(November, 1S>66), p. 3- 
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Variables in Teaching Situations 

It is apparent however, that the teaching process is not the 
only factor which contributes to the behavior of nupils within a given 
class. If this were the case, the behavior (including achievement) of 
all pupils in a given class would be the same. There are aspects of 
every teaching situation over which teachers have no direct control 
but which modify or otherwise alter the Influence which they atteii5)t 
to assert. These aspects are in many cases attributable to the 
social and cultural backgrounds of the pupils. Generally, p\Q)ils 
from families which value education and which have provided them 
with a multitude of enriching educational experiences outside of 
school tend to attain higher levels of academic achievement. 

There are also factors in the physical teaching situation 
which affect responses of pupils to teaching. The time of day, the 
weather, or the type of facilities and materials available can make 
certain types of facilities and materials avaU.afcle can make certain 
types of teaching behavior more or less suitable and can affect the 
learning of the pupils. 

Of prime significance in a teaching situation is the nature 
of the objectives toward which the teaching is directed. Objectives 
vary from school district to school district, from school? to school 
within a district, from classroom to classroom within a school, and 
even from pupil to pupil within a classroom. As objectives vary, 
the importance of different types of teaching behavior and different 
types of pupil behavior will vary also. 
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ThuS; information on teaching "which does not account for the 
context in which the teaching takes place is apt to be ambiguous. 

Certain types of teaching behavior or teaching outcanes may be desirable 
in certain situations and undesirable in others. 

The Approach for Reducing Ambiguity 
Proposed in the Present Stu^ 

To reduce ambiguity^ it is necessary insofar as is possible, 
to make explicit the implicit factors which contribute to uncertainty 
in interpreting information on teaching. The approach advocated in 
the present study is singly to collect such additional infomation and 
to consider it when analyzing data on teaching performance or teaching 
outcomes. The additional data would relate mainly to the criteria 
"Which are relevant. Such information would also relate indirectly to 
situation factors which would presumably be taken into consideration 
"when criteria are established. 

Because of the volume of the information to be processed, "with 
numerous clerical problems of sorting and collating likely to ensue, 
the proposed system was conceived with electronic data processing 
equipment in mind. The files of infomation would be maintained on 
magnetic tapes, discs, or other de"vices and the analyses would be 
carried out via computer. 

Conceptual Framework for This Study 
• As stated earlier, the purpose of this study is to delineaoe 
the conditions under which statewide programs for measuring and evalu- 
ating teaching can yield the most useful infomation for Implementing 
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state educational leadership in the education of teachers, certification 
of teachers, and accreditation of schools. It is not anticipated 
that such information vould he used at the state level to make decisions 
relative to the quality of individual schools or to the quality of work 
done hy individual teachers. It would he used to evaluate and to ijnprove 
state policies affecting large numbers of teachers or schools. 

The basic atm of this study is to outline a system for assembling 
at the state level usable information on teaching. Usable information on 
teaching is defined herein as that information which shows the rela- 
tionship (congruity or discrepancy) between desired teaching processes 
and/or products and observed teaching processes and/or products. If 
good teaching is defined as behaving in a certain manner under certain 
conditions, information which is usable will show congruity or dis- 
crepancies between the criteria and the observed behavior in those 
situations. 

Guidelines 

There are a number of established practices which have been 
accepted as sound by both professional educators and the public which 
they serve. Those stated below are examples. They will be treated 
as guidelines and, hence, honored as proposals are developed within 
the ciirrent study. 

1. The state should not pre-empt local initiative in determining 
local educational objectives. It should, however, stimulate 
local authorities to articulate and Justify their objectives. 

2. The state should not dictate to professional educators the 
procedures which they should follow or the specific outcomes 
which they shoiad seek in their efforts to implement local 
objectives. It should, however, stimulate members of the 
profession to articulate and Justify their practices. 
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The present study atten5>ts to outline approaches throu^ which 
a state can fulfill its responsibilities without usurping the domains 
of educational policy makers and professional educators within local 
school districts. In general, it proposes that the state assume leader- 
ship in developing systems through which (l) objectives and criteria 
can be stated more clearly, (2) outcranes and procedures can be described 
and measured more objectively, and (3) measures of outcomes or pro- 
cedures can be ccmpared directly with objectives (or criteria) to 
yield usable information. 

Me asurement. Evaluation and 
^ediction 

The raw information on teaching which is collected constitutes 
measurements. A measurement of teaching is defined herein as the end 
product obtained throu^ analyzing the teaching with a standard classi- 
fication system (i.e., a scale). This classification system can relate 
to aspects of either the teaching process or the teaching product. 

Measurements, in themselves, are neutral. They merely describe 
what has happened in terms of a given classification system. In order 
to determine whether what has happened is good, the measurements must 
be related to an appropriate set of criteria. This process, the com- 
paring of evidence (measurements) with criteria, constitutes evaluation. 
»Ph-tR concept has been explicated i.n some detail by Yildirim. Since 

it is possible for criteria of effective teaching to vary, it is pos- 

✓ 

sible that one set of measurements will represent good teaching when 




^iHirijn, pp. 49-76. 
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conrpaxed with one set of criteria and poor teaching when compared with 
another set. 

Prediction is an atteu^t to estimate in advance what given 
measurements will he. Programs for the certification of teachers con- 
stitute an application of prediction. These programs are operated on 
the assumption that the criteria which they en^loy are effective in 
discriminating between those who, if allowed co teach, would tend to 
receive satisfactory ratings and those who would not. Certificates 
are then issued only to those in the first group. 

The present study deals first with measurements by reviewing 
and discussing measures of teaching which have been or could be applied. 
It then treats evaluation by discussing wa^"s in which these measures 
might be related to evaluative criteria (or ways in which evaluative 
criteria might be related to these measures). Such a process would 
thus provide a basis for testing the validity of predictors now employed. 
This study does not, however, set out to validate or invalidate any 
specific standards or programs. The purpose herein is to set forth 
a framework or general system which can be employed to facilitate the 
use of information on teaching for implementing state educational 
leadership. 

Some Specific Questions 

The proposed system is designed to include information which 
is relevant and accessible for answering several inportant questions 
relative to teacher education, teacher certification, and school 
accreditation. This information would also be usable for a number of 
other purposes. The questions in the following list are illustrative: 
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(1) What criteria do individual colleges think are relevant for 
evaluating the teaching of graduates of their programs? Are 
these the same for all graduates? For all graduates in given 

teaching fields? 

(2) How does the teaching done hy graduates of different types of 
teacher education programs measure up against the evaluation 
criteria considered appropriate hy the teacher education 
institution? By the local r hool districts? 

(3) What criteria do individuao. .chool districts or schools judge 
to he relevant for evaluating teaching in their classrooms? 

Are these the same for all or most teachers within a school 
district? Within a school? For all or most teachers of 
certain grades or subjects? 

( 4 ) How does the teaching done hy teachers in Individual schools 
measure up against the evaluative criteria which are set forth 
as appropriate hy the local school district? 

(5) HoWdo the criteria specified for different segments of the 
teaching population (e.g.^ beginning teachers, music teachers, 
sixth grade teachers, etc.) compare? 

(6) How do eval\iation‘ criteria selected hy school districts and 
evaluation criteria selected hy colleges con^^are? 

(7) Is there more than one distinctive pattern of expectations 
for teachers in a given field? 

(8) What are the inservice education needs of various segments of 
the teaching population? 

(9) What are the relationships between teacher status characteris- 
tics^ *^viz., the things which are considered in recommending a 
haccalauiFeate candidate for a teaching certificate or in 
issuing a teacher certificate) and teaching (evaluated in terms 
of criteria established hy local school districts)? 

(10) What relationships exist between school status characteristics 
(viz., the things which are given consideration in recommend- 
ing~a school for accreditation) and the teaching and learning 
which takes place in those schools? 



^Teacher status characteristics which are considered to he 
necessary antecedents to effective teaching constitute presage 
criteria. 
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Outline of the Study 

Chapter II reports a questionnaire survey of statewide programs 
for evaluating teaching. It also contains descriptions of the programs 
in three states (Hawaii, Pennsylvania and Florida) in which the state 
prescribes procedures for evaluating teachers. In Chapter III, general 
approaches for evaluating teaching are reviewed. Chapter IV proposes 
a system for obtaining information on teaching which could be used to 
Inrplement state educational leadership. Summarizing and concluding 
remarks are presented in Chapter V. 
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CHAPTER II 



STATE PRESCRIBED PROCEDURES FOR EVALUATING TEACHING 

Comprehensive information on the quality of teaching within 
a state can contribute significantly to the process of state educa- 
tional leadership. However, as pointed out in Chapter I, any effort 
to evaluate teaching is, from the staxt, plagued with complex problems. 
There are many different concepts regarding the nature of acceptable 
criteria for evaluating teaching. Possibly because of this, research 
which should be expected to provide guidance has made few significant 
contributions to current practices in evaluating teaching. In 
addition, there are numerous factors within the context of the teaching 
situation which effect both the processes and products of teaching. 

Educational literature contains virtually no mention of 
statewide efforts to evaluate teaching. Only two references contain- 
ing information on this subject were found in the literature: one by 

Rasley and one by Kelley. Rasley,^ through a survey of teacher evalua- 
tion programs, found that Hawaii, Florida, and Pennsylvania require 

^Charles W. Easley, "TER, A Pillar of Strength: A Curreut__ 

View of Teacher Effectiveness Rating as Used in the Public Schools 
(a Survey Resesirch Study, Division of Education in the Graduate 
School.,- Southern Illinois University, Edwardsville, I966), p. 39 . 
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that teachers he evaluated. An article hy Kelley^ describes the state- 
wide progrem for evaluating teachers in Florida. 

Since it was not clear from Rasley*s study whether states 
other than the three cited were polled in his survey, a new question- 
naire survey was carried out. The results (identiced with Rasley*s 
from the standpoint of states included) are reported below. 

A Survey of Statewide Teacher Evaluation Programs 
Purpose .— The purpose of the survey was to find out which 
states have within the past ten years carried out a program of state- 
wide evaluation of teaching and to ascertain the purposes which those 
evaluations were Intended to fulfill. The evaluation with which this 
survey was concerned is defined as an attempt by a state government 
or its agent to secure a composite assessment of the qu8LLlty of the 
teaching which is taking place within the state. 

The questionnaire .— The questionnaire was a one-page instru- 
ment containing seven questions. A copy is included as an appendix. 

The first three questions asked (a) whether a statewide evaluation of 
teaching has been conducted within the last ten years, (b) whether 
such evaluations are conducted annually, and (c) when the three most 
recent evaluations were conducted. The final four questions asked 
about the puj.poses for conducting each of the three most recent 
evaluations. The purposes specifically mentioned were (a) to recognize 
and promote meritorious teaching, (b) to recognize and eliminate in- 
competent teaching, and (c) to accumulate research data. 

T. Kelley, **Teacher Svaluation," Florida School Bulletiii> 
Vol% 24 (December, 19^)> PP* 23-28. 
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Population . - -The population surveyed included the state 
director of teacher education and/or certification in each state. 

The names "were obtained from the 1965“66 Roster of State Directors of 
Teacher Education and Certification .^ Only one questionnaire vas 
sent to each state; the recipient was the one whose title iuoplied 
that he would be most likely to have responsibility for a statewide 
evaluation program in his state, if one existed. 

Proced ures and returns . -"A cover letter was written and signed 
by W. Cecil Golden, Director, Division of Teacher Education, Certi- 
fication and Accreditation, Florida State Department of Education. A 
copy of this letter is included as an appendix. A stamped return 
envelope, addressed to Golden, was enclosed with the questionnaire and 
the letter. Individually typed follow-up letters, signed by Golden, 
were sent in September and a g a i n in November to all persons in the 

t 

population who had not yet returned their questionnaires at those 
times. Enclosed with each follow-up letter was an additional copy 



of the questionnaire and a stamped retuxii envelojje. 



Eventually, com- 



pleted questionnaires were received from all states. 

Follow-up of the questionnaire survey .— Those persons who 
indicated that their states did carry out statewide teacher evalua- 
tion programs were contacted by mail, by telephone, or in person to 
obtain additional information as to their procedures and purposes for 
these programs. Through this approach, it was found that some of the 



Foster of State Direct ors of Teacher Education and Certifi- 
cation (Washington, D. 0^ The National Commission on Teacher Educa 
tion and Professional Standards, 19 ^ 5 ) • 
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states vhich responded affirmatively to the first question did not, 
in fact, have an evaluation program which would fall within the limi- 
tations of the definition en 5 )loyed in the present study. The approach 
in these states was usually to conduct surveys to determine either 
the educational levels attained hy their teachers or the extent to 
which certification requirements were being met. 

Results of the questionnaire survey .— Questionnaire returns 
indicated that only three states— Pennsylvania, Hawaii, and Florida— 
prescribe procedures for evaluating teachers to be used throughout 
the state. The programs in these states are described in the subse- 
quent sections of this chapter ^ This information was obtained throu^ 
contacts following the questionnaire survey. 

The Application of State Prescribed Teacher 
Ev^uation Procedures in Pennsylvania^ 

In Pennsylvania, a state prescribed rating system is in effect) 
employing the fom depicted in Figure 1. The rating of teachers is a 

p 

statutory requirement designed to eliminate incompetent teachers. 

The rating foim is prepared by the Department of Public Instruction. 

The responsibility for rfiiting teachers rests with the County or District 
Superintendent, but he may delegate this authority to a lower adminis- 
trative or supervisory officer. Unsatisfactory ratings assigned by 

^e description in this section is based upon information 
furnished by Harris W. Reynolds, Education Evaluation Advisor, Pennsyl- 
vania Dep^tment of Public Instructicxi. 

^School Laws of Pennsylvania, Article XE, Section 1123, as 
cited by Reynolds in a personal letter to W. Cecil Golden, August 19, 

1966. 
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iigJmtifi’l^ iK 



Pig. 1— ."Temporary and Professional Employes’ Rating Card" 
Used in Pennsylvania (front and "back sides). 
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such officers are official, however, only when signed hy a commissioned 
officer of the Commonwealth of Pennsylvania a County or District 

Superintendent, Assistant Superintendent or Associate Superintendent). 
Permanent records of a-li ratings for each teacher are maintained in 
the teacher’s school district. All ratings of a teacher must he trans- 
mitted to him at his request. An unsatisfactory rating will he 
transmitted automatically. 

Both beginning and tenure teachers are evaluated. A teacher 
who enters the teaching profession in Pennsylvania must serve a two- 
year, or sometimes three-year, probationary period. During this time 
he is termed a "temporary professional employe" and is entitled to 
neither contract nor tenure status. "Temporary professional employes" 
are evaluated twice annually. At the termination of two years of 
service with "satisfactory" ratings the teacher is eligible to become 
a "professional employe" with both contract and tenure status. 
"Professional employes" are evaluated once each year. A "professional 
employe" may be dismissed only for cause; if the cause is inccmpetency, 
the rating form provides the necesseiry record. 

The ratings assigned teachers are also given consideration in 
the renewal of teaching certificates. Only those teachers with 
ratings of "satisfactory" are eligible to have their certificates 
renewed. 

Since the rating form is intended to identify and eliminate 
incompetent teachers and since a rating of "unsatisfactory" on this 
form will, in fact, prevent a teacher from achieving tenure status, 
considerable care is exercised in prescribing procedures which must 
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be followed when assigning an ’’unsatisfactory" rating: The rating 

form must, of course, he completed and signed properly. In addition, 
substantiating evidence must be provided, stating "in plain English 
the specific conditions or instances upon which the rating is based. 
On the other hand, if a teacher is rated "satisfactory," it is not 
necessary to provide, exther on the form or elsewhere, informatxon 
which indicates the teacher's relative strengths or weaknesses. 

As for the value of the Pennsylvania program. Department of 
Education officials from that state feel that it is effective in- pro- 
viding a means of eliminating teachers who have glaring weaknesses. 
These are teachers who are quite obviously in^equate. It is not de- 
signed, however, to provide infomatlon for use in making educational 
decisions (other thtui decisions as to whether, in specific cases, 
teaching certificates should be issued). 

Statemde Teacher Evaluation in Hawaii' 

The plan of school organization in Hawaii is unique in that 
the state, itself, operates all schools, rather than delegating this 
authority to local districts. Thus, Hawaii's approach to teacher 
evaluation is in many ways similar to one i-rtiich might be taken in a 
large local school district. 

I I I ■ ifc I ■ * " ' ‘ 

^The description in this section is based upon information 
furnished by Minoru Ezaki, Staff Specialist, Personnel Development, ^ 
and Harry Chang, Staff Specialist, Recruitment and Placement, Hawaii 
State Department of Education through personal correspondence and 
consultation, I 966 and 1967* 
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In Hawaii, evaluation of teachers is required hy law.^ The 
stated purpose of such evaluation is to in5)rove instruction in che 
schools of the State. To fulfill this purpose, separate forms and 
procedures for the evaluation of probationary teachers and the evalu- 
ation of tenure teachers have been developed. In both cases, the 
primary responsibility for evaluating the teacher rests with the 
principal. Both the principal and the teacher who is evaluated must 
sign the form. (The signature of the teacher is not meant to inply 
that he concurs with the evaluation but merely that he is aware of 
it.) Following up on recommendations made to the teacher is primarily 
the duty of the principal. However, in cases where the rating is 
unsatisfactory, the district office also takes action. 

Reports of the evaluations are placed on file in the State 
Department of Education where they are deposited in folders of indi- 
vidual teachers and become part of the teachers* peimanent personnel 
records. Copies of the completed forms are also kept in the files of 
the district superintendent and of the principal who made the evalua- 
tion. An additional copy is given to the teacher who is evaluated. 

Every teacher beginning service in Hawaii must serve a pro- 
bationary period of four consecutive semesters. This period may be 
extended to five years. During each semester, the principal pays 
several visits (four or five are suggested) to classes of each proba- 
tionary teacher. After each Visit, the principal conrpletes the form. 



Revised Laws, of Hawaii, Section 38-38, as amended (1965), cited in 
"Personnel Memo No. 318 (1965-66)" (Honolulu, Hawaii; State of Hawaii, 
Department of Education, October 22, I965, mimeographed). 
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“Re.port on Principal’s Classroom Visit" (Figure 2), and reviews it 
with the teacher. Three copies of the fom are filled out: one copy 

to he retained hy the principal, one to he given to the teacher, and 
one to he forwarded to the district office. Near the close of the 
semester, the principal prepares a report (Figure 3) summarizing 
his prior observations. This report is also reviewed and signed hy 
hoth the principal and the teacher. It is prepared in quadruplicate, 
with the original being submitted to the Office of Personnel Services, 
State Department of Education and with the remaining copies being 
distributed as were the classroom visit reports. When the summary 
report following the fourth semester of teaching is submitted to the 
Department, proceedings to grant tenure are initiated. If all four 
reports are satisfactory, tenure will be granted automatically. If 
they are not, the case will be investigated. This will involve con- 
sultation between the principal, the district superintendent, the 
State Department of Education personnel officers, and, ?rf necessary, 
with the teacher also. 

Tenure teachers are to be evaluated at least once per year. 

The evaluation report has two parts: one to be filled out by the 

teacher which deals with conditions affecting teacher performance 
(Figure k) and one to be filled out by ihe principal which deals 
with the teacher’s professional qualities (Figure 5)* Both forms are 
prepared in quadruplicate and signed by the teacher and the principal, 
with copies being distributed to the State Office of Personnel Services, 
the teacher, the principal, and the district office. 
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5, Coomenta: 



Sat la factory or better 
Not Satiafactory 




(ConiliM* »n »Witr tldt tf noocsiary) 

(roaohop'itlcnttiufe d«09 not ncooMarlly a«m 
•OMpioto dfiPotaont on thp part of ths> toaoher,) 



Teacher^ a Signatura 



Date 



principal 'a Signature 
Data 



Fig. 

Probationary 



2— ’’Report on Principal’s Classroom Visit” Used for 
Teachers in Hawaii, 
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Notet This fotffl may be re - 
produced at the 8chool « 

STATE OF HA\IAII 
DEPARTMENT OF EDUCATION 
PERSONNEL DIVISION 



PROBATIONARY TEACHER EVALUATION SUHI-IARY REPORT 
(To be filed with Pcraonncl Division— for instructions, 
refer to Principals' Circular No» 1902) 



TEACHER'S NAME (Last first) SCHOWi 



SUBJECT and/or GRADE 

REPORT FOR: First Year Second Year 

1st Prob. Sem. (Intern- 3rd Pt'obi Seitii 

ship or Regular) Prob, Sem, 

2nd Prob, Sem _______ 



POINTS OF STRENGTH: 



POINTS OF NEARNESS: 



SUMMARY: 



1st, 2nd 
or 3rd 
Semester 
Report 



4th 

Semester 

Report 



Is performing satisfactorily or better as , 

a probationary teacher at this time, I i 

Should not be offered new contract, — — IZ3 

Should be dismissed Immediately, nn 

I cannot make a Judgment at this time, (A 

judgment must be made in the 4th semester,) I I 

It is my carefully considered professional opinion that this 
teacher: r— — » 

Should be granted tenure, I I 

Should not be granted tenure, I 1 



(Teacher's signature does not necessarily mean 
complete agreement on the part of the teacher,) 



TEACHER'S SIGNATURE DATE 



PRINCIPAL'S SIGNATURE DATE 



Pig. 3— "Probationary Teacher Evaluation Report" Used in Hawaii, 
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STATE OF HAMAH S/lV/fiS 

DEPARTMENT OF EDUCATION 
HONOLULU, HAWAII 



TENURE TEACHER- EVALUATION REPORT 



Year 19 - ") 



TEACHER 






Number of years 
with Deot. of Educ. 




Latt| 


First 


Hri,Mrti,Hltt 








Grade or 


SCHOOL 






Subject 



PART ONE - CONDITIONS OF WORK AFFECTING TEACHER PERFORMANCE 
(To be Filled in oy Teacher and Olacuaaed with Principal) 
This St tn opportunity for ihu tcnchor to Hotorlbo oondltlont that arfooi hit 
toaohinc* You aro Invited to mnko coirmontt that apply to you* Rotum to your 
I Rrlnoipal boforo tho ond of tho 1st quarter in Novdnbor (4 ooplci)« 

> - — - ^ 

(Check or Fill in Information) 



1. CLASSROOM ENVIRONMENT 
At Student! 



TEACHER* 3 COMMENTS AND SUGGESTIONS 
(Comment on Itema that Apply to You) 



1. Ability: Fait ; Average^ ; 

Slo w 1 Mixed Group t 

2t Stability of Enrollment: 

■ Stable ; Tram lent . 

3t Special Problema: 

Gifted ; M.R. } Payaically 

Handicapped ' ; Emotionally 
Diaturbed ; Other Problema 



B. Availability of Instructional Materlala 
Such aa : 

Books Workbooks Supplies, Equiprent, 
A.V. Aids. 

Available t Not Readily Available t 



C, Physical Facilities Such as : 

Furniture, Light, Ventilation, Storage, 
General Conditions in Room .and Building: 
Adequate . Inadequate ‘ ' ♦ 



Fig. 4— "Tenure Teacher Evaluation Report," Part I, Used in 
Hawaii. (Continued on next page.) 
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- 2 - 



TKACHER 



Last, 



First 



II. TEACHING LOAD 

A. Teachtne Rcsponptbllltles 



TEACHER’S COMMENTS AND SOGGESTIOMS 
(Comment on Items that Apply to You) 



1. Number of teaching periods 

2. Total number of students 

enrolled . 

3. Number of subject preparations 

4. Special Teaching Programs: 

Yes No 



B. Non-Teaching Responsibilities 

1. Grade level and Committee Meetings 

2. Special Assignments such as: 

School Programs; A.V.; J.P.O.; 

Yard Duty; Lunch Duty; Book 
Evaluation; Curriculum Work. 

3. Other non-teaching duties such as: 
Clerical work; First Aid and Other 
Health Problems; Parent Conferences. 

III. TEACHER PLACEMENT 



Appropriate placement in accordance with 
his training and experience. 

Yes_j No 



IV. OTHER CONDITIONS AFFECTING TEACHER 

pf:rformance 

Such as: Classroom Interruptions; 

Coimnunity Service; Money Raising 
Activities; Special Requests of Teachers 



Teacher's Signature 



Date 



COMMENTS BY PRINCIPAL; 
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TEACHER 

Last» 



- 3 - 

SCHOOI. 

First 



YEAR 19 • 19 



PART TWO - PROFESglOKAL QUALITIES 
(To be Filled In by Principal and Discussed with Teacher) 
Due: By May 1 (4 copies) 



I. KNOWLEDGE OF SUBJECT M4TTER 

A. Demonstrates a knowledge of subject 
matter being taught 

1. Shows evidence of adequate back** 
ground in subject area 

II. PRESENTATION OF WTERIAL 

A. Uses effective procedures and 
materials appropriate to the 
maturity level, Interest and 
ability of students 

B. Uses different or varied materials 
and techniques In Instruction 

111. ORAL AND WRITTEN EXPRESSION 

A. Uses standard American English 
and communicates effectively 

IV. LESSON ORGANIZATION AND PLANS 

A. Plans for effective teaching 

B. Conn^ders children's needs and 
Interests 

C. Provides for varied and 
stimulating experiences 

D. Provides. for student Involvement 
In the learning process 

E. Provides for varied re-lnforcement 
of skills 

V. CLASSROOM CLIMATE 

A. Maintains classroom climate con- 
ducive to learning 

B. Promotes spirit of participation 
and willingness to work 

C. Exercises good judgment In main- 
taining discipline 

D. Holds respect of students 

E. Shows respect for students' 
accomplishments 



EVALUATOR'S C01M5NT3 AND SUGGESTIONS 
FOR IMPROVEWNT 

(Comment Only on Significant Features) 



Fig. 5— "Tenure Teacher Evaluation Report," Part II, Used in 
Havraii. (Continued on next page.) 
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- 1 * - 



'li;Af'.llKR 

l.ast, First 



VI. C1.ASSR00M H A NAGEMENT 

A. Shows good organization and 
preparation 

B. Administers routines o.ifcctlvely 

C. Prcvldes for satisfactory physical 
classroom conditions 

VII. ATTITlIpr.S AN[> WORKINC; RELATIONSHIPS 

A. Accepts constructive suggestions 

B. Effect i'ely implements the sugges- 
tions for improvement 

C. Likes people and works well with 
students, fellow teachers, admin- 
istrators and parents 

VIII. ORGANIZATION OF SCHOOL DETAILS 

A. Keeps adequate school records 

B. Meets expected deadlines 

C. Knows and follows school procedures 
and rules 

IX. PROFESSIONAL IMPROVEMENT 

A, Keeps up with professional trends 
and techniques in education 

B. Improves himself professionally— e .g. 
workshops, institutes, college 
courses, reading, travel, xesearch 

PRINCIPAL’S SUMMARY ; 

Satisfactory Not Satisfactory 

COHHliNTS IW rUTNCIPAL; (If uddltlonal opneo Is neoded, ottsch snolhor shoot.) 



EVALUATOR’S COMMENTS AND SUGGESTIONS 
FOR IMPROVEMENT 

(Comment Only on Significant Features) 



COMMENTS BY TEACHER; (If nddlttmial sf/ioo Is noodod, nttoch onothor shoot.) 



(Tcochor’s slciintdro duos not ncncssnrily Jndleato opprotsl but ncroly that ho Is awsro of evaluation) 



Teacher's Signature 



Date 



• Principal's Signature 


R rC r j V LDate 

f.l '|l*» * / n 

AUG..1 AJ9BB 




Evaluator’s Signature & Title 


Date 




(If other than Principal) 


STAtt otPt. 0^ tuucAliON 
ftomna 






The staff of the Office of Personnel Services of the Hawaii 



State Department of Education feels that the evaluation program for 
probationary teachers is effective in eliminating incompetent teachers. 
About one percent of the new teachers are rated unsatisfactory, and 
consequently, are released immediately or at least are not rehired for 
a second year. The staff members think that the evaluation program, 
along with an incentive program which provides increased salaries for 
those teachers who volimtarily undertake a planned program of profes- 
sional improvement, is effective in improving instruction in the 
state. They feel also that the tenure teacher evaluation program has 
served to stimulate concern for evaluating and improving instruction 
where such concern did not previously exist. 

Statewide Teacher Evaluation in Florida 
The specific purposes which the Florida teacher evaluation pro- 
gram is ejtpected to fulfill have not been defined. The Florida 
statutes^ do require, however, that each certified school employee 

, I 

(tfe&eher, administrator, etc.) be evaluated annually and that this 
evaluation be placed on file in the State Department of Education. 

Two additional copies of the evaluation form are completed: one for 

th'^ county office and one for the files of the local school. Evalua- 
tions of teachers are normally completed by principgds and counter- 
signed by county superintendents. In the State Department of Educa- 
tion, the evaluations become part of the teacher personnel files. They 



^Statutes of Florida (1965), Section 231* 25* 
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are available for inspection only by State Department of Education 
personnel and by county school personnel on official business. 

A staiidard form to be used for the evaluations has been adopted 
by the State Board of Education (Figure 6). The •f'^rm is designed 
so that it caji be read with an optical scanner. The ratings are thus 
transferred automatically to punched cards. The punched card data 
are transferred to magnetic tape for retrieval and analysis via 
computer. This has made possible the summarizing of ratings for indi- 
vidual schools, for counties, and for the. entire state. Statewide 
Summaries of ratings for the years 1963-^^ and 1964-65 presented 
in Tables 1 and 2. 

Data from the Florida teacher evaluation forms have been used 
by a number of Florida State University students in their graduate 
thesis and dissertation projects. Studies of this type which h£\re 
been completed to date made use of an earlier version of the evaJ.uation 
form. This version differs from the present in that it contains more 
items and employs a scale with five response positions, rather than 
three. The nature of the items on the new and old forms is the same, 
however. 

1 

Wurzbach studied the relationship between principals* ratings 
and three status factors: degrees held, teaching experience and 

county of eirployment. He found a significant amount of variance in 

X 

Edward G. Wurzbach, "A Tb,ree-County Study of Teachers* State 
Evaluation Scores in Relation to I’heir Experience and Type of Degree” 
(unpublished M.S. thesis, Florida State University, Tallahassee, I962). 
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Fig. 6- -Florida Teacher Evaluation Form 
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TA3IE 2. —Ratings a$sl®ied to 49,977 Florida teachers for the 1964-65 school year using the Florida 

Teacher Evaluation Form 
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the ratings attributable to differences in collegiate degrees held. 

His ■ procedures might be questioned, however, as he apparently summed 
the composite scores for each section of the rating sheet in order to 
obtain the figures which were used in the analysis. To justify such 
a practice, it is necessary to make tenable the assumption that all 

items on the foim can be assigned equal weight. 

Carter^ used the state evaluation form to distinguish between 

high .and low merit-rated junior college teachers. He then studied the 
relationships between these ratings and personal, educational, and experi- 
ence factors. He found few significant relationships. He, like Wurzbanh, 
also used composite scores as his measure of teacher competence. 

Gerlock^ compared the ratings given professionally and pro- 
visionally certificated first year teachers. He did separate item- 
by-item analyses and found that those teachers meeting professional 
certification standards scored significantly higher on some items, 

particularly those relating to teaching skill. 

Four additional studies dealing with factors which might warrant 
consideration in interpreting data collected with the Florida evaluation 
form have also been coii 5 )leted. McTeer^ investigated the hypothesis that 



^Fl-etcher Fairwick Carter, "Selected Aspects 

Preparation and Prior Experience of Hi^ aM '^^orida 

College Teachers in Florida" ^unpublished Ph.D. dissertation, Florida 

State*^University, Tallahassee, 1964). 

^Donald E. Gerlock, "An Analysis of Administrators* Evaluations 
of Selected Professionally and Provisionally Certificated Seconds^ 
School Teachers" (unpublished Ed.D. dissertation, Florida State Uni- 
versity, Tallahassee, 1964) . 

^John Hugh McTeer, "A Study of the Relationship of Tether- 
Principal Likenesses and the Principals* Ratings of Teacher Effec- 
tiveness" (unpublished Ed.D. dissertation, Florida State University, 

Tallahassee, 1963)* 
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teacher-pj^liicipQl likenesees are a factor in teacher ratting. He assumed 
that the more similarities vhich existed between the teacher and the 
principal, the higher would be the teacher *s rating. Factors which he 
studied include age, grade point average, highest degree held, and amount 
of professional work as an undergraduate. He found some significant 
correlations, but in no case did the likeness factors account for a sub- 
stantial amount of the variance, i^ome shortcomings of his study are 
that’ he apparently combined scores on scale items and did not account 
for difference in the general level of scores assigned by different 
principals. 

Packer^ was Interested in the sensitivity of -bhe instrument to 
•'self-others acceptance** which he deemed an essential trait for an effec- 
tive teacher. He found very little relationship between this construct 
and the principals' ratings and concluded that either the principals were 
not sufficiently familiar with the teaching situations or they were not 
appreciative of "self-others acceptance" as a determinant of effect 
teaching. 

Two recent studies by Daniel eri 5 )loyed the current version of the 
Florida teacher evaluation fom. The first employed the analysis of 
variance technique to determine whether certain factors could account 
statistically for differences in ratings assigned to a teacher on any 
items on the form.^ The factors tested were (a) subject or grade being 

•^Morton Alfred Packer, "A Study of the Relationship Between Teachers* 
Self-Others Acceptance and the Principals* Ratings of These Teachers" (un- 
published Ed.D. dissertation, Florida v^tate University, Tallahassee, 1^). 

%. Fred Daniel, "A Catalog of Analysis of Variance Pilot Stxjdies" 
(unpublished research report, Multl^^State Teacher Education R^oject, 

State Department of Education, Tallahassee, Florida, 1966)* 
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taught (i.e., Is th^re any difference between ratings assigned to teachers 
of one grade and those assigned to teachers of another grade?), (b) 
evaluator (i.e., Is there any difference between the level of ratings 
assigned by one evaJ.uator and those assigned by others?), (c) certificate 
rank (i.e., education level), (d) number of different preparations which 
the teacher must make, (e) sex of the principal, and (f) age of the 
principal. The only one of these factors which contributed significantly 
to the variance in ratings was the evaluator effect. This can be inter- 
preted two ways: either some principals tend to rate their teachers 

higher than do other principals, or some principals have better 'teachers 
in their schools than do others. A subsequent series of studies has 
been begun to investigate this problem further. The value of such efforts 
is limited, however, because technical deficiencies of the evaluation 
form place limitations on interpretations which can be made from statis- 
tical manipulations o;: the data therefrom. This problem is discussed 
later in greater detail. 

A second study en^loying the present form deals with the objec- 
tivity of ratings assigned by principals.^ Objectivity is defined as the 
extent to which independent ratings of one teacher completed by two dif- 
ferent evaluators are in agreement. In this study, ratings by assistant 
principiils and by supervisors were con^ared with those submitted by 
principals. These ratings are summarized in Table 3* I't was found in 
both the sample of assistant principals and the sample of supervisors 
that the principal and the other rater agree about two-thirds of the 

hi. Fred Daniel, "A Study of the Objectivity of the Florida Teacher 
Evaluation Form" (unpublished research report, Multi-State Teacher 3i]duca- 
tion Project, State Department of Education, Tallaiiassee, Florida, ISoj). 




TABLE 3.— Comparisons of ratings .assigned to the same teachers using 
the official Florida Teacher Evaluation Form 







Comparison of supervisors* ( 

and principals' rstings | 

1 


lomparison of assistant 
;>rlncipals' snd principals* 
ratings 






•S .pervisors* ratings 
higher than principals* 


Supervisors * ratings 


lower than principals* 




1 

Supervisors' and prin- 
cipals' racings same' 


Assistant principals* 
rav:ings higher than 
principals* 


Assistant* principals* j 
ratings lower than 


principals* 


Assistant principals* 
and principals* 
ratings same 


I. PERSONAL QUALIFICATIONS 


M 


% 


rN. 




r. 


N % 


N 


% 




% 


N 


% 


A. 


Is Healthy and Emo- 
tionally Stable 


49 


2fi 


30 




17 


1 

95 _55 


62 


18 


^4 


18 


222 


84 


B. 


Is Nest snd Well 
Groomed In Apocarsnce 


Ul. 








n 


107 61 


34 


1? 


70 , 


20 


227 


65 


C. 


Thinks Logically and 
Makes Practlcsl 
Decisions ^ 


28 


If} 


27 , 




10 


J.1B 68 


45 


Ifi 


H2 


23 


213 


81 




Is Accurate 


27 


16, 


32 




IS- 


1 14 66 


45 


13 


70 


22 


227 


85 


E- 


Is Punctual _ 


38 


22 


40 




23 


95 __55 


5$ 


io 


74 , 


21 


217 


8.1 


F. 


Takes Necessary and 
Appropriate . Action 
On Hia Own 


28 


16 






20 


109 63 


45 - 


13 


82 


24 


221 


64 


G. 


Is Dedicated to 
His Profession 


27 


16 


38 




23 


107 62 


58 


17 


M— 


18 


2?9 . 




II. RELATIONSHIPS KITH OTHERS 


























■ A. 


Is Respected by Pupils 


43 


25 


28 




16 


102 . 59 


44 


1 J 


ZO. 


20 


237 


68 


B. 


I8 Responsible and 


36 


21 






16 


lt)9 63 


.22— 


-IL 




24 


228 


-JI5. 


C. 


Is Friendly, 
Understanding', 
Svir.DnthctlC-.- . . 


28 


16 


27 




16* 


118 68 


38, 


11 


96 


28 


215 


_8I 



o 
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TABLE 3 .—Continued 





Comparison o*^' 
and prlncloals. 


supervlsorn' 
' ratings 


Comparlsot 

prlnclpalt 

ratings 


1 of assistant 
»' and principals' 




Supervisors • ratings 
higher *:haa principals 




Supervisors ' ratings 
lower than prmcipals' 




Supervisors * and prin- 
cipals* ratings same 




Asslscaac principals' 
racings higher chan 
principals' 




-Asslscanc principals' 
racings lover chan 
principals' 




•Assistant principals* 
and principals* 
ratings same 




M 




N 


7. 


N 


7. 


N 


7 . 


N 


7. 


N 


X 


D. Is Morally Uprleht* 


0 


.0 


0 , 


, 0 


171 


100 


.0 


Q 


H 


1 


344 


99 


E. Is Professionally 
Ethics! 


. 1 


1 


9 


? 


161 


J2A 


4 


1 


B 


2 


CD 

1 m 

i 




III. TEACHING ABILITY 
A. Knows Subject Matter 


25 


15 




19 


109 


fifi 


49 


14 


55 




242 


(y, 


B. Takes Action To 
Improve Himself 




15 


47 


27 


99 


58 


53 


n 


66 


11 


231 


66 


C. Uses Instructional . 
Materials and Lessen 
Plans Bffectlvclv 


31 


mi 


1 22 


13 


118 


69 


.52 


12 


.61^ 


18 


227 




D. Develops Pupil 

Interest and Eagerness 
to Learn 


35 


20 


26 


-li. 


JIL_ 


65 


57 


17 


75 


IL 


216 


, 61 


E. Maintains Pupil 

CnnhrAl 


37 


22 


27 


16 


106 


62 


60 


17 


75 


21 


214 


-JJL 


F. Uses Material In- 

rtiiiHit FAtdftr 


29 


IB 


ZQ 




109 


-69 


46 


14 


64 


19 


_22SL 


. 68. 


IV. WOULD YOU RECQWKHD 
THIS TEACHER * 

FOR EE-EMPLOYMEirrt 


1 . 






1 


155 


, 9fl 


2 




16 


-5. 


315 


94 


TorrAi. 


530 


16 


I 4 ?? 


15 


2213 


68 


m 


17 


1179 


18 


4592 


70 



*An IteM vlth only tvo possible responses: yes nnd no. 
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time, (with the dis trihut ions of ratings observed, agreement could usually 

be expected about half of the time by chance.)^ 

Evaluating the Florida teacher evaluation program is difficult 

since there is no specific purpose which the program has been designated 

to fulfill. It might be assumed, however, since the evaluation forms are 

dev:‘sed for machine processing and since routines have been established 

to tabulate and summarize the data, that there has been some intention to 

secure ccmprehensive information relating to the quality of teaching in 

the state. This is information which might be used in developing ways to 

inprove the educational program. Further evidence of such an intention 

can be found in the proposal for a "multi-state project to inprove teacher 
2 

education," submitted to the U. S. Office of Education in I965. It was 
proposed that the Florida State Department of Education use its teacher 
evaluation data along with other data to deteiroine needed services in 
the areas of pre-service and inservice teacher education. 

When viewed in this framework, the Florida teacher evaluation pro- 
gram displays striking inadequacies. The technical deficiencies of +he 
instrument are one example. The extreme skewing in the distribution of 
ratings assigned on the form limits the utilization of parametric statis- 
tical techniques when analyzing the data. Thus, inferences drawn from 



^Probabilities of agreement between principals* ratings and assist- 
ant principals* ratings on a given item were calculated by multiplying the 
proportion of the sample rated in each category by principals times the pro* 
portion rated in the same category by assistant principals. These products 
were then summed to obtain the probability of agreement on that item. 



2 

This project has been funded under Title V of Public Law 89-10 
(Elementary and Secondary Education Act of I965) and is now called the 
Multi-State Teacher Education Project. The proposal was submitted by 
the Maryland State Departn^nt of Education, the project administrator. 
Other states in the Project are Florida, Michigan, South Carolina, Utah, 
Washington, and West Virginia. 
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the data using standard scientific procedures must be interpreted with 
caution. The lack of homos cedacticity which can result from the skewing 
often makes it inadvisable to calculate Pearson product -moment coeffi- 
cients of correlation to provide indices of the relationships between 
ratings received and other variables. The limited number of steps on 
the rating scale virtually eliminates the possibility of using this index. 

The principal handicap which is imposed by a rating scale with a 
limited number of steps results from restrictions upon the amount of in- 
formaticsQ which the scale can provide. The optimum number of scale 
points is that which makes maximum use of the observer’s discriminative 
powers. This point is reached irtien the ratio between true variance and 
error variance is maxinrum. Guilford^ reviews a number of studies which 
deal with the optimum number of steps to be used in a rating scale. He 
concludes that the number is usually greater than seven and may, in cer- 
tain situations, be as high as twenty- five. The Florida foim employs 

scales with three, and in some cases two, steps. 

The Florida evaluation form also appears to have some conceptual 
deficiencies. Particul^ly notable is the great emphasis placed upon 
general attitudes or personality traits which are deemed to be conducive 
to effective teaching and the small amount of emphasis upon behaviors 
which definitely fall into the latter category are, "Uses instructional 
■ materials and lesson plans effectively, " "Develops pupil interest and 
eagerness to learn," and "Maintains pupil control." Following the dis- 
tinction which was made in Chapter I, only these items fall within the 
realm of evaluation^ the others constitute predictors. 

P. Guilford, Psychometric Methods (New York: McGraw-Hill 

Book Con^jany, Inc., 193 ^) > PP* 289-29I. 
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To smnmarize, it appears that, because of deficiencies of the 
evaluation instrument, the Florida teacher evaluation program can 
provide very little infomation which can be used to in5)rove education 
in Florida 



Concluding Statement 

It appears that neither a review of the educational literature 
nor a study of current practices can provide direct guidance in de- 
veloping a program for collecting and analyzing information on 
teaching to facilitate state educational leadership. Only two refer- 
ences were discovered which make mention of the subject. Two of the 
three state programs for evaluating teaching which are now operating 
(Pennsylvania and Hawaii) are not designed for obtaining the type 
of data which could be used for implementing state educational leader- 
ship. The third program (Florida), which appears to have been con- 
ceived for such a purpose, displays some serious inadequacies. 

Thus, to' proceed in developing a system for collecting and 
analyzing the desired data, it is necessary to turn elsewhere for 
assistance. In the present study, general approaches to the evalua- 
tion of teaching are discussed. Then, a system for assembling and 
analyzing data obtained with these approaches is proposed. 




CHAPTER III 



APmOACHES TO THE EVAUJATION OP TEACHING 

Evaluation was described in Chapter I as the process of com- 
paring evid nee with criteria. The amount of evidence which could be 
gathered through the obs'.rvation of teaching processes and products 
is overwhelming. Every syllable uttered and every movement made by 
teacher and pupils could be recorded on tape or film. In addition, 
sensing devices could be used (as has been done with astronauts in 
outer space) to monitor and record various physiologicEd actions and 
reactions of teachers and pupils. Thus a prime goal In developing 
methods for evaluating teaching is to reduce the evidence to an 
amount which is manageable and to Include in the portion retained 
that which is significant. Whether or not a given observation or 
rating system will in practice prove adequate depends \^on its adapta- 
bility to the criteria for evaluation which have been adopted, for it 
is. the criteria which determine which evidence is significant. 

The criteria dictate not only the type of evidence to be 
sought, but also the specific approach or approaches which are appro- 
priate for obtaining the measures which will constitute the evidence. 
It follows then that an approach for collecting evidence cannot be 
decided i^on until criteria for evaluating teaching are selected. 

47 
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Selecting Criteria 

The types of criteria which have been used in evaluating 

teaching have been classified by Mitzel^ in three categories : product 

criteria, process criteria, and presage criteria. Product criteria 

comprise the outcomes toward which teaching is directed: changes in 

behavior of pupils. Such changes are often declared to be the 

ultimate criteria of teacher effectiveness. Writers expressing this 

opinion include the Committee on the Criteria of Teacher Effectiveness 

appointed by the American Educational Research Association, Ackerman, 

k 

and Yamamoto. 

It is maintained by many persons, however, that factors other 
than the influence of the teacher contribute significantly to changes 
in pupil behavior and, thus, it is not possible to evaluate the work 
of a teacher solely in terms of the achievement of his pupils. In an 
effort to account for extraneous influences and still use changes in 
pupils as a criterion, modified approaches for deriving pupil gain 
scores have sometimes been used. These include achievement quotients, 

"Slarold E. Mitzel, ^'Teacher Effectiveness," Encyclopedia of 
Educational Research, ed. Chester W. Harris (New York: The Macmillan 

Compar^, 1960^, pp. 1482-85 . 

^H. H. Remmers, et al ., "Second Report of the Committee on 
Criteria of Teeuiher Effectiveness," Journal of Educational Research, 
Vol. 46 (May, 1953), p. 642. 

^Walter I. Ackerman, "Teacher Coupetence a-nd Pupil Change," 
Harvard Educational Review, Vol. 24 (Fall, 1954), p» 274. 

\aoru Yamamoto, "Evaluating Teacher Effectiveness: A Review 

of Research," Journal of School Psychology, Vol. 2 (January, 1963), 
p. 6l. 



residual gain scores, and scores obtained throu^ variations on the 
residual gain procedure.^ However, in a large portion of the re- 
search dealing with teaching effectiveness, product criteria have 
not been employed. Medley and Mitzel state that the proportion of 
studies in which neasui^s other than pupil growth have been used as 
criteria approximates 90 per cent. Only nineteen out of I 38 studies 
in a summary ccn5)iled by Barr^ employed pupil gain as a criterion. 

Process criteria are those types of teacher behavior believed 

to be desirable, at least in given situations. They constitute 

an obvious alternative for consideration when problems of securing 

and interpreting information on pupil gains are deemed sufficiently 

serious to preclude the use of product criteria. In the domain of 

the local administrator or other instructional leader (as contrasted 

, 4 

with the domain of the researcher), process criteria are of particu- 
lar significance. Here is where decisions relative to retention, 
promotion, salary, supervision, inservice education programs, and other 



William J. Ellena, Margaret Stevenson, and Harold Webb (eds.), 
Who*s a Good Teacher? (Washington, D. C. : American Association of 

School Administrators, I 981 ), p. 15* 

2 

Donald M. Medley and Harold E. Mitzel, "The Scientific Study 
of Teacher Behavior," Theory and Research in Teaching, ed. Amo A. 
Bellack (New York: Bureau of Publications, Teachers College, Columbia 

Ikiiversity, 1963 )^ P* 83 . 

O 

Arv±l S. Barr, "The Measurement and Prediction of Teaching 
Efficiency: A Summary of Investigations," Journal of E3{perimental 

Educaticn, Vol. I 6 (June, 19^ PP* 203-283 . 

^^Ellena, Stephenson, and Webb (eds.), pp. 5-6. 
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types of staff development must be made. These are ongoing decisions 
which cannot wait until evidence becomes available which would permit 
teaching to be evaluated in terms of ultimate achievements of 
pupils . 



Presage criteria are made up of those traits and experiences 
which are thou^t to be fundamental to certain facets of beaching per- 
formance (which can be evaluated employing process criteria) or to 
achieving certain outcomes (which can be evaluated enploying product 
criteria). Thus, presage criteria function primarily as predictors. 
Their value is determ.ined by their efficiency in pre4icting which 
persons will teach effectively and which will not. Studies have, 
nevertheless, been undertaken enploying presage variables as criteria, 
rather than predictors. .Also, it is not uncommon to find on rating 
forms items such as, ”Is healthy," or "Is well versed in subject 
matter." These items relate to traits which are commonly thought to 
be antecendents of creditable teaching. They do not, in themselves, 
comprise measures of teaching. 

The various specific factors which can be incorporated into 
criteria for evaluating teaching are numerous. The product, process, 
and presage categories each enconpass a multitude of possibilities. 

The Taxonci.y of Educational Objectives^ illustrates several categories 



benjamin S. Bloom (ed. ), Taxonomy of Educational Objectives, 
Handbook I; Cognitive Domain (New York: David McKay Company, Inc., 

195^) j David R. Krathwohl, et al.. Taxonomy of Educational Objectives, 
Handbook I I: Affective Domain (New York: David McKay Company, Inc., 

19 ^ 4 ). 
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•which might he used in developing product criteria, oince measures 
in each of the categories can he taken after varying lapses of time, 
the potential number of specific product criteria which can he 

adopted is coii5)ounded immediately. 

Process criteria can also encompass several different dimeinsions. 

Brown^ has developed instinmients to measure the extent to which prac- 
tices of teachers agree or disagree with practices endorsed hy the 
John’Dewey philosophy. Flanders^ has devised a series of categories 
to measviro the nature of the verbal influence which teachers attempt 
to impose on piQ)lls. His categories consist of types of direct or 
indirect influence. "Using student ideas" is an example of Indirect 
Influence and "giving directions" is an example of direct influence. 
Smith^ has devised a system for analyzing in different terms the 
teacher's use of language. His categories comprise "logical opera- 
tions;" defining, describing, and conditional inferring are examples. 

A group at Stanford University^ has analyzed teaching into several 

^oh Burton Brown, "Bringing Philosophy into the Study of 
Teacher Effectiveness," Journal of Teacher Education, Vol. 17 
(^ring, 1966), pp. 35-^ • ■ 

^Edmund Amidon and Ned A. Flanders, The Rol e of the Teache:^ 
in the Classroom: A Manual for Understand i ng and Improving Teach^^ 

Classroom Behavior (Minneapolis: Paul S. Anldon and Associates, 19o3 ) • 

^B. Othanel Smith and Milton Meux, "A Study of the Logic of 
Teaching," Cooperative Research Project Numloer 258, Trial Edition 
(Urhana, Illinois: Bureau of Educational Research, College of Educa- 

tion, University of Illinois, I962). 

^James M. Cooper, "A Performance Curricijl'jm for Teacher Educa- 
tion," The Second Annual Florence B. Stratemeyer Lecture read before 
the Annual Meeting of the Association for Student Teaching, Chicago, 

February I6, 19^7 • 
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separate sub-skills or tasks and has devised criteria for evaluating 
each of these. Examples of the skills are making assignments, monitor- 
ing in-class assignments, small group work, and Introducing a unit. 

Presage criteria can also include a multitude of different 
factors. The range is restricted only by such limits as mi^t be in- 
herent in the ability of man to categorize human traits and experiences. 

Are Some Criteria More Worth- 
while than <^' ers? 

With so many possible criteria ava,ilable, it is reasonable to 
ask whether some criteria mi^t not possess intrinsically greater merit 
than others. Are there guidelines which mi^t help professional edu- 
cators to deteimine which criteria would be most acceptable in a given 
situation? 

According to Rabinowitz and Travers, there are available no 
objective procedures for identifying acceptable or unacceptable criteria 
of teaching effectiveness. Effective teaching does not exist independ- 
ently but is an artifact created when an independent or collective 
value judgment is made. The authors state that, "No teacher is more 
effective than another except as someone so decides and designates. . . . 
The ultimate definition of the effective teacher does not involve 
discovery but decree."^ Rabinowitz and Travers suggest that the place 
to start in developing criteria is with the goals which’ the teaching is 
expected to accomplish. The teaching which contributes to the attainment 

William Rabinowitz and Robert M. W. Travers, "Problems of De- 
fining and Assessing Teacher Effectiveness," Educational Theory, Vol. 3 
(July, 1953), P. 212. 
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of these goals is thus considered effective. The goals are, of course 

I 

established on the basis of value judgments. Travers and Rabinowitz 
eventually go so far as to declare that pupil growth is the most 
sensible criterion of effective teaching. However, this recom- 
mendation must be considered in light of their earlier statement. 

Ryans^ also believes that no type of criterion of effective 
teaching possesses intrinsic goodness. The worthiness of any given 
set of criteria is dictated by the values of the specific culture 

which the teaching is intended to se].*ve. 

2 

Yildirim, however, maintains that it is an overstatement to 
say that there is no basis for validating a criterion outside of what 
someone decrees to be in 5 )ortant. He feels that such arbitrariness 
serves to affirm that the judgment of one person is as good as that 
of any other .person? that the judgment of the layman is equally as 

valid as that of the professional. 

Yildirim! s position is that knowledge and understanding of the 
situation in which the criteria are to apply is necessary in order to 
the Optimum criteria. He admits that the criteria result 
from a value judgment, but this judgment must also have an empirical 
base. The persons establishing the criteria must maintain contact 

with the realities of the teaching situation. Criteria without such 

* \ 

a base are likely to be capricious. 

'^vid G. Ryans, Characteristics of Teachers: Their Description, 

Comparison and Appraisal ’^Washington, D. G. : American Council on Educa- 

tion, I960), p. lb. 

^Cemal Yildlrtm, "An Analytic Model for Evaluation of Teacher 
Competence" (unpublished Ph.D. dissertation, Indiana University, 
Bloomington, 19 ^ 1 ), pp. 95-102. 
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It was reasoning along the lines of the Yildirim argument that 
led to the adoption of one of the guidelines set forth in Chapter I 
of the present study: the one declaring that the state should not 

dictate to professional educators the procedures which they should 
follow or the specific outcomes which they should seek in their 
efforts to implement local educational objectives. Recommendations 
regarding evaluative procedures o' criteria should he mediated hy 
knowledge resulting from familiarity with the situations to he affected 
hy the recommendations. 

Some Clarifications Regarding 
the Present Study 

It was specified in Chapter I that this study deals with measure- 
ment and evaluation of teaching hut does not deal with the problems of 
predicting the quality of, teaching which might he expected in any given 
circumstance. Therefore, it does not deal with presage criteria since 
their relevance is principally in the area of prediction. Because of 
this, systematic approaches for measuring antecedents of teaching (i.e., 
presage or status factors) are not treated in the present chapter. It 
should he noted, however, that if the traits which conprise presage 
criteria are described in behavioral terms and thus can he observed 
in the classroom, the presage criteria at once become process criteria. 
Hence, the distinction between presage and process criteria is not as 
decisive as it might appear at first encounter. 

Likewise, the distinction between process and product criteria 
is not always obvious unless everything that the teacher does is 
considered process and everything which the pupils do is considered 
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product. However, when pupils at the direction of the teacher line 
up on two sides of the room for a spelling contest, it seems more 
reasonable to consider this activity as part of the teaching process 
and to consider the degree of accuracy with which the pupils spell 
the words as part of the teaching product. 

The distinction is not critical, however. The point is that 
criteria can relate to things that happen before the teacher begins to 
teach, to things that happen in the course of a teacher *s teaching, 
and to things that happen after the teaching is conqoleted. This 
chapter deals with procedures for evaluating teaching in tenns of those 
things which happen during or after a teacher’s teaching which, on 
the basis of the adopted criteria, axe considered significant. 

Collecting Evidence and Compaxing it with Criteria 

As the above discussion indicates, the criteria for evaluating 
teaching which can be selected con^rise a varying assortment. Likewise, 
severed, different approaches which can be employed to gather evidence 
and to compare it with criteria axe available. Not all of these approaches 
are equally applicable to any given type of criterion. Also., not all 
are equally suitable for assembling information which can be used in 
Implementing state education leadership. The remaining portion of this 
chapter contains descriptions of a diversity of approaches. The descrii-'- 
tions are Intended to provide an indication of the range of choices 
available. The utility of various approaches, in terms of their poten- 
tial for contributing information .for implementing state educational 
leadership, is treated more explicitly in Chapter IV. 
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In the subsequent discussion, procedures for collecting evidence 
and canparing it with criteria are conceived as varying along three 
dimensions. The dimensions represent (l) the situation in which the 
evidence is collected, which varies from a "nor’ situation to a 
"constructed" situation, (2) the agent responsible for coding and re- 
cording the information, which has "self" (i.e., the teacher or pupil 
performing) at one extreme and "others" at the opposite extreme, and 
(3) the relationship between collecting evidence and comparing evidence 
with criteria (i.e., evaluating), which can vary from "evaluation while 

t 

observing" to "evaluation independent of observation." These dimen- 
sions are described below in greater detail. They are also depicted 
graphically in Figure 7. Any evaluative procedure can be classified 
in the three-dimensional space of this diagram. The letters A, B, C, 

D, E, F, G, .and H identify extreme points on each of the dimensions 
in the three-dimensional space.. Following further explanation of 
the dimensions, approaches to evaluation are discussed and. couched 
in terms of these extremes. It should be kept in mind, however, that 
there are also infinite intermediate positions between the extremes. 

The first dimension relates to the situation in which the 
evidence is collected. At one pole of this dimension is the normal 
day-to-day teaching situation or the normal life situation in \^iich 
an individual might be found subsequent to being taught. The latter 
type of situation would be of interest if teaching were being evaluated 
in terms of certain types of product criteria. At the opposite pole 
of this dimension is a situation constructed to elicit certain types 
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Fig. 7— Graphic representation of system for classifying evalu- 
ation procedures along three dimensions. 

of specific evidence. A test or examination is an example of a con- 
structed situation. This, of course, represents the approach nomally 
employed to measure pupil learning, the desired product of teaching. 

Constructed or standardized situations are seldom, if ever, 
used in evaluating the teaching performance of inservice teachers. 

It seems possible, however, that if criteria for effective teaching 
■ were ’stated in terms of specific teaching tasks (as described by 
Cooper^ for example) that standardized situations could be constructed 

to measure objectively and to evaluate the performance of teachers on 

2 

these tasks. The classroom simulator designed by Kersh mi^t serve 
^Cooper. 

^Bert Y. Kersh, **C]lnssrocm Simulation: A New Dimension in 

Teacher Education,” Title VII Project Number 886, National Defense 
Education Act of 1953 (Monmouth, Oregon: Teaching Research, Oregon 

System of Hi^er Education, 1983)» 
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as a prototype for a mechanism which could he used in presenting stimuli 
in the constructed situation. 

The second dimension relates to the agent who codes the evidence. 
At one pole, the evidence is coded by the person performing the 
behavior. This would be the teacher if the evidence relates to 
teaching performance, or the pupil if the evidence relates to teaching 
products. In the usual objective test, the established procedure 
for responding is designed so that the responses are recorded in 
coded form by the person taking the test. . In an essay test or a 
performance test of almost any type (e.g., a music contest, an ice 
skating contest, a debate tournament, or a job interview), the evidence 
which will be compared with the evaluation criteria is coded by some- 
one other than the performer. 

The third dimension relates to the point at which the actual 
evaluation takes place; that is, the point where the evidence is 
compared with the criteria. At one pole of this dimension, the evalua- 
tion takes place during. the observation process. At the opposite pole, 
the collection of the evidence and the evaluation of the evidence are 
carried out independently. With rating procedures, the recorded 
rating constitutes an evaluation. In other words, the person making 
the observation compares what he sees with the applicable criteria 
and then records the resiilts of the con5)arison. 

When the collection of evidence and the evaluation of the 
evidence are carried out independently, a procedure is developed where- 
by the observer describes, in narrative or quantitative terms, what 
has taken place. This description is limited to those aspects of the 
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performance of the teacher and/or pupils which are relevant to the 
criteria. An example of this approach is the inberaction analysis 
system which has been adopted to the classroom by Flanders^ and others. 
The evaluation of the teaching takes place after the observation record 
has been con5)iled. With such a system, the evaluation can be done by 
persons other than those who conducted the observation. The same 
data can also be analyzed employing different criteria, providing, 
of course, that the alternate criteria encompass the same elements 
which were recorded in the observation. 

Sel f Reports and Self Evaluatiai 

Self evaluation is the fundamental procedure by which teachers 

can iiiprove their teaching. Considering self reporting and self evalu- 
ation in terms of process criteria, area A in Figure 7 represents the 
self evaluation carried out by individual teachers In day-to-day teach- 
ing situations. Area E represents a procedure whereby self reports are 
prepared by a teacher and evaluated Independently, possibly with the 
aid of a consultant. This type of practice seems most likely in a 
situation where a teacher is having a special problem and seeks out- 
side help. Areas B and F parallel A and E except that the measures 
are taken in a constructed or standardized situation. Such a situa- 
tion would be possible only in an institution >*ich has provisions for 

simulating classroom situations. 

If teaching were being evaluated In terms of product criteria, 

self reports could be solicited from pupils. This would take place 




^Amldon and Flanders. 
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in a, constructed situation unless it happened that the desired data 

were emitted spontaneously or as part of some other activity, fhe 

data obtained from the pupils could be in the form of judgments 

(areas A and B) or in the form of descripblve infomation or responses 

to test questions (areas E and F). The latter type of information 

would be evaluated independently. 

In general, self reports from teachers which describe or 

evaluate the teaching which has taken place in their classrooms would 

be of limited use in implementing state educational leadership. 

According to Biddle, the cognitive systems normally used by teachers 

for thinking about classroom activities are not adequate for objectively 

describing classroom situations.^ It seems, however, that, if the 

teacher were sufficiently familiar with the criteria for evaluation, 

this need not be the case. Nevertheless, the problem of assuring the 

reliability or objectivity of the evidence remains. As Travers points 

2 

out, an individual observing himself is a biased observer. The sedf- 
report approach need not be eliminated categorically, though, as there 
might arise certain situations in which such an approach might prove 
efficacious, possibly when used in conjunction with other procedures. 

^Bruce J. Biddle, ’'The Integration of Teacher Effectiveness 
Research, ” Conteniporary Research on Teacher Effectiveness, ed. Bruce 
J. Biddle and William J. Ellena (New York: Holt, Rinehart and Winston, 

196^0^ pp. 272-8. 

2 

Robert M. W. Travers, An Introduction to Educational Research 
( 2 d ed; New York; The Macmillan Con5)any, 19 ^ 5 )> P* 272 . 
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For evaluating teaching in terms of product criteria the 
testing approach (area F) is normally employed. This can be classified 
logically as another form of self-reporting. Since this is the 
approach typically used for evaluating pupil achievement, it is 
reasonable that data so obtained be adapted for evaluating teaching. 

r 

Testing is discussed at greater length in a subsequent section of 
this Chapter. 



Rating and Rating Scales 

The most singular characteristic of rating as an evaluation 

% 

method is that the evidence is ccmpared with the criteria during the 
observation process and only the comparison (i.e., the rating) is 
recorded. Thus, rating is represented in Figure 7 at points A, B, C, 
and D. At points A and B, rating is performed by the person 
whose perfomance is beint ---ired. Thus, if process criteria were 
being employed, the teacher would perform a self evaluation and if 
product criteria were being employed, the pupil would evaluate his 
own skills. In areas C and D, the rating is assigned by an observer. 



Area D represents the usual situation when process criteria are 
employed, in which an observer cones to the classroom and rates the 
teaching performance. Area C represents the same type of rating but 
the teaching performance takes place in a standardized or constructed 
situation. If product criteria were employed, an observer would rate 
the performance of pupils in a special test situation (area C) or in 
their normal environment (area D). 
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Rating scales are "by far the most widely used devices for 
evaluating teaching performance for both research and administrative 
or supervisory pn. ,’oses. At least in the case of administrative and 
supervisory situations, this condition is likely to persist. The 
evidence which must be reviewed to determine whether or not teaching 
is effective is invariably extensive and subtle with numerous complexi- 
ties which are difficult to catalog in advance. Because of this, quasi- 
mechanical or objective procedures for evaluating teaching have 
usually not been considered acceptable, and raters themselves have 
been required to perform the function of reducing the data to that 
which is significant and comparing this evidence with the relevant 
criteria. As a result, summarizing and processing of evidence takes 
place enbirely within the "black box" and only conclusions are avail- 
able for scrutiny. Hence, neither the data reduction process nor the 
evaluation process can be examined. If a case developed in which two 
"experts" evaluating the same teaching pr^. /ided different evaluations, 
it ■would be a matter of speculation as to whether the discrepancy 
resulted from their selecting different evidence to process or from 
their applying different criteria in evaluating (unless, of course, 
the evaluators were available for questioning). 

Fortunately, the evaluation process employing rating scales 
need not be so mercurial as the foregoing implies. Travers^ suggests 
that the stability of results so obtained can be controlled, thus 



'Ibid., pp. 



222-224. 
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making the results meaningful by controlling both the type and quantity 

of information to be processed and the processing itself. This can 

be dbne by providing sufficient descriptive material with the rating 

form to orient the user, by constructing a rating instrument coii 5 )osed 

of specific rather than general scales, and by constructing the 

individual scales carefully. Practices recommended by Travers which 

should result in better scales include (l) defining with precision 

several points on each scale, (2) restricting each scale to a limited 

range of well-defined behavior, ( 3 ) varying the end of the scale 

which represents "good,” and (4) avoiding the use of words such as 

"average" to represent the middle range of a scale. Discussions of 

technical considerations in rating scale development and the litera- 

1 2 

ture relating to their \ise are presented by Guilford and Remmers. 

An obvious technique for improving the reliability of ratings 
involves the training of raters. Such training could consist of a 
thorough orientation into the type of evidence which is to be con- 
sidered significant and the type of criteria which are to be employed 
in analyzing it. This would be followed with practice in enploying 
the scale Including opportunities for comparing and discussing the 
ratings assigned. Practice sessions can be repeated until the desired 
level of reliability is reached. Garrison, who has been involved in 

^J. P. Guilford, "Rating Scales," in his Psychometric Methods 
(New York: McGraw-Hill Book Con 5 )any, 1954), pp. 263-301. 

H. Remmers, "Rating Methods in Research on Teaching," 
Handbook of Research on Teaching, ed. N. L. Gage (Chicago: Rand 

McNally and Company, 1963), PP« 329-378 • 
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two large-scale research projects employing a rating form for apprais- 
ing teaching perfonnance, obtained reliability coefficients of .73 to 
.85 with a training period in which trainees observed and evaluated 

four lessons^ and coefficients of .53 to .89 with a training period 

2 

in which trainees observed and evaluated only one lesson. 

In employing information collected with rating forms to imple- 
ment state educational leadership, data obtained with any carefully 
developed form administered with carefully established procedures is 
acceptable, as long as the form utilizes normative, rather than 
ipsative, scales. Normative scales make comparisons between individuals; 
that is, they use a norm as their model. Ipsative scales make con5)ari- 
sons within individuals; that is, they compare one aspect of the per- 
formance of one individual with other aspects of the performance of 
that same individual. Data obtained with ipsative scales are not in- 
tended to be grouped. The combining of ratings from several different 
observations of several different teachers would, of course, be 
necessary in deriving information intended to affect statewide 
policies. 



^Harry L. Garrison, "Evaluation of Teaching and Learning," 
(unpublished Ed.D. disseration, Stanford University, Stanford, 
California, 1964 ). 

2 

F. Herbert Hite, "Effects of Reduced Loads and Intensive 
Inservice Training Upon the Classroom Behavior of Beginning Elementary 
Teachers," Cooperative Research Project No. 2973 (Olympia, Washington: 
Washington Superintendent of Public Instruction, I966). 




65 



Separating the Observation Process 
from the Evaluation Process 

In recent years research on teaching has been moving away from 
the study of teaching effectiveness as a global concept. Rather than 
rating teachers along an effectiveness -ineffectiveness scale and then 
attempting to find the correlates of these rat togs, researchers have 
been studying relationships between several variables associated with 
teaching. These consist of traits of teachers, dimensions of teaching 
performance, characteristics of teaching situations, traits of pupils, 
and dimensions of pupil performance. The approach has been first to 
describe objectively and to measure these variables and then to look 
for relationships which exist between them, whether or not these 
relationships bear any immediate direct connection to any spec, ,’ic 
concept of effective teaching. The relationships studied are ones 
which may have been confounded when a more general approach to research 
on teaching was followed. The general assumption underlying this newer 
approach is that when interrelationships between variables affecting 
teaching are understood better, teaching will be understood better and 
the various combinations of conditions which contribute to effective 
teaching will be more easily recognized. 

An outgrowth of this movement has been the development of 
observation procedures which can be used for describing objectively 
various, aspects of teaching processes or products. These make it 
possible to separate the collecting of evidence on teaching from the 
evaluating of that teaching. This approach offers the potential for 
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more objective evaluations by enabling the observer to concentrate 
more directly on collecting evidence and by making the evidence avail- 
able -for review both before and after the process of comparing it with 
the criteria has been carried out. This approach also allows more 
than one set of criteria to be applied to a given set of evidence and 
allows different elements of the evidence to be combined and studied 
in different ways. 

In Figure the approach in which observations are separated 

i 

from ratings is represented in areas E, F, G, and H. Area H represents 
the situation in which an observer would collect data on the perform- 
ance of a teacher or a pupil employing a standardized observation 
procedure; the data would be evaluated subsequently. Area F repre- 
sents a confStructed situation, most likely to occur when evaluating 
pupil learning, in which the pupil records his coded responses; these 
will be evaluated later. Area G represents a constructed situation 
with someone other than the performer recording the responses and 
area E represents a situation in which a performer records data on 
his own performance and evaluates it later. 

The first step in developing an observation system is to de- 
termine what aspects of teacher or pupil performance are of interest. 

The second step is to categorize these elements so that they can be 
% 

objectively reported by an observer or to locate a category system 
already in existence which can be adapted to a given evaluation pro- 
gram. Several such systems are already in \ 3 se„ The Classroom Inter- 
action Newsletter^ conducted a survey of category systems in use and 

^Classroom Interaction Newsletter, Vol. 1 (May, 19 ^ 6 ), pp. 1-2. 
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received 315 questionnaires from persons wno are using them in their 
research or teaching activities. Fifty- three different observational 
systems were represented among these. The most widely used systems 
were the Observation Schedule and Record (OScAR)^ by Medley and 

2 

Mitzel and the interaction analysis system developed by Flanders. 

Medley and Mitzel,^ in a general article dealing with observa- 
tion methods, described two different types of observation schedules. 
The first is called a category system. It consists of an exhaustive 
list of categories all dealing with one aspect of behavior. Every unit 
of behavior observed is classified into one of the categories. The 
completed observation record sl.ows the total number of behavior units 
obs ed and the number classified in each category^ The Flanders 
interaction analysis system is an example of the category type. It 
Includes seven categories, which describe the types of verbal influ- 
ence exercised by the teacher. They are (l) accepts feeling, ( 2 ) 
praises or encourages, (3) accepts or uses ideas of student, (k) asks 
questions, (5) lectures, (6) gives directions, (t) criticizes or 
justifies authority. The first four constitute indirect influence 
and the final three constitute direct influence. There are two addi- 
tional categories which deal with student talk, one for student talk 

^Donald M. Medley and Harold E. Mitzel, "A Technique for Measur^ 
ing Classroom Behavior," Journal of Educational Psychology, Vol. 49 
(April, 1958), pp. 86-92. 

■ p 

Amidon and Flanders. 

^Donald M. Medley and Harold E. Mitzel, "Measuring Classroom 
Behavior by Systematic Observation, " Handbook of Research on Teaching, 

pp, 288-303. 
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in response to the teacher and one for student initiated taJ.K', and 
a final category called silence or confusion bo account for every- 
thing not covered previously. The Flanders system is designed so 
that not only the number of behavior units falling into each categf>ry 
can be studied but also so that the sequence of categories can be 
analyzed. The Flanders approach need not be restricted to the 
Flanders categories, but may also be used with such other catrgor;Ies 
as might be relevant in a given situation. 

The second approach to constructing an observation schedule 

described by Medley and Mitzel is called the sign system. With this 

system, a list of behaviors which may or may not occur is compiled. 

The observer then tallies those behavior units observed which meet 

the categoi’y definitions. It is not assiuned that all behaviors which 

occur will be recorded. An example of this procedure is found in the 

1 

Teacher Practices Observation Record which is designed to dctermin<i 
the extent to which the practices of a teacher .:oincido with those 
advocated by the experimental philosophy of John Dewey. To use the 
schedule, a thirty-minute observation period is divided into three 
ten-minute segments. The observer checks which of the sixty-two 
listed teaching practices occ\at in each of the three segments. 

Tlie observation prtjcedui’e developed by Medley and Mitzel for 
use with the OScAR combines the category and sign system. A 

■^Bob Burton Brown, Teacher* s Classroom Behavior (Gainesville, 
Florida: Teacher Competence T^esearch Project, Cdlege of Educal.i<'n, 

University of Florida, undated). (Tills is a group of instriunents f(\r 
use in evaluating a teacher. ) 
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thirty-minute observation perisji is divided into six five-minute seg- 
ments. The first, third, and fifth segments are spent tallying on a 
list (i.e., using the sign system) the types of activities, groups, 
materials, and behaviors which are .observed. During the second, fourth, 
and sixth segments, the expressive behavior of the teacher is recorded 
using a category'' system. The categories' are (l) non-verbal pupil- 
supportive, (2) pupil-supportive, (3) pijoblem- structuring, (If) mis- 
cellaneous, ( 5 ) directive, (6) reproving, and ( 7 ) non-verbal reproving. 
The type of subject, dealt with in each of the five-minute segments is 
also recorded. 

The category system offers the advantage of accounting more 
thoroughly for behavior along a given dimension. To employ it, how- 
ever, the number of categories must be limited so that the observer 
can keep them all in mind simultaneously and categorize observed be- 
havior instjantly. The sign system allows for a wider range of be- 
haviors to be included. It does not, however, provide information as 
to the relative frequency of the behaviors. Both systems are appli- 
cable to programs for the evaluation of teaching employing either 
process or product criteria, provided, of course, that the relevant 
behaviors are defined and included in the list of categories used. 

The training of observers is imp6rtaht with sign and category 
observation systems just as it is with rating systems. Therefore, the 
remarks about training of observers which were made earlier app3.y here 
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Testing 

If product criteria are to be used in evaluating teaching, the 
place of testing is obvious. On the other hand, if process criteria 
are selected, the value of constiucted situations for evaluating 
teaching is virtually mexplored. There seems a possibility, however 
(as stated earlier), that if criteria for effective teaching were 
stated in terms of specific tasks, standardized situations could be 
developed to sample the behavior of teachers in performing the 
individual tasks. > 

Areas B, C, P, and G in Figure 7 represent constructed or test 
situations. In area B, the performer (either teacher or pupil) pre- 
pares a self-report while performing. In area C, the observer evalu- 
ates the performance using a rating scale. In' area P, the person 
being tested codes evidence for evaluation later; this is the normal 
objective testing situation. In area G, an observer records evidence 
for evaluation later. Areas C and G represent the usual performance 
testing situation. 

Teachers and administrators are well aware of procedures for 
examining pupils to assess their learning. However, they have been 
generally unwilling to use the results of these assessments as a basis 

for evaluating teaching for several reasons which have been previously 

% 

discussed. Their reasons might be summarized with two statements: 

(l) there are many factors which act before, during, and after a 
teacher’s teaching which affect the amount of learning which takes 
place within any given Individual, and (2) the tests which are available 
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may not represent the f\iH range of objectives toward which the edu- 
cational program is directed. 

The second criticism has been answered to a certain extent by 
the two volumes of the Taxonomy of Educational Objectives^ which demon- 
strate that examination items can be written to yield information in 

areas which have previously received little attention. An approach 

2 

for dealing with the first problem has been presented by Daniel. 

In its present form, such merit as the idea might possess is of 
theoretical value only, because of the problems inherent in imple- 
menting the proposal. The general idea is to determine whether 
pupils taught by a given teacher have achieved as much as they could 
reasonably be expected to achieve. The procedure consists of first 
developing the educational goals or criteria and stating them in 
operational terms. T?hese goals would be assigned weights based upon 
their judged importance. (The combined value of all weights would 

/ 

sijm to one.) This would mean that the teacher would have in mind 
definite outcomes for each pupil which he teaches and also be aware 
of the relative importance of each. Then, on the basis of all avail- 
able information (past performance, aptitude, current motivation, etc.) 
an estimate of the amount of achievement which could be expected for 
each pupil would be recorded. At the appropriate time subsequent to 

^Bloom; Krathwohl. 

^K. Fred Daniel, ’ "Effective Teaching and Educational Values, " 
paper read before the Eighteenth Annual Meeting, Philosophy of Educa- . 
tion Society, Southeastern Region, Tuscaloosa, Alabama, February 4, 

1966. 
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the teaching, the achievement of the pupil .in all relevant areas would 
he measured and con 5 )ared with his predicted or expected achievemenl-.. 

The ratio of each observed achievement to each expected achievement 
yould he calculated and multiplied hy its respective weight* The 
weighted rating would then he summed to provide an index of teaching 
effectiveness* Thus, a teacher whose pupils achieved as much as they 
were expected to achieve would earn an index of 1*00* A teacher 
whose pupils achieved more than they were expected to achieve would 
have an index greater than 1.00* A teacher whose pupils achieved 
less than they were expected to achieve would have an index of less 
than 1*00. The principal difficulty in applying such an approach to 
evaluating teaching would no doubt ccme in trying to arrive at valid 
estimates of e 2 ^>ected achievement. 

Concluding Statement 

Tliis chapter has set out (l) to indicate the primacy of criteria 
in establishing an evaluation program, (2) to indicate the wide range 
of choices which can he' considered in adopting criteria of effective 
teaching, and (3) to indicate the range of approaches available for 
collecting evidence and evaluating teaching* The fact that well con- 
ceived teacher evaluation programs may exist in the schools of a state 
■ does not, however, mean that the state is automatically provided with 
the information which it needs to evaluate its activity s in teacher 
education, certification, and accreditation, and to begin immediately 
to exercise more effective educational leadership* A system must 
first be developed for collecting and analyzing the information which 




sound local evaluation programs can provide. An approach for doing 
this is outlined in Chapter IV. 



CHAPTER IV 



AH APPROACH TO ORGANIZH'IG AND ANALYZING INFORMATION ON 
TEACHING TO IMPLEMEITO STATE EDUCATIONAL LEADERSHIP 

As stated in Chapter I, the purpose of this study is to delineate 
the conditions under which statewide programs for measuring and evalu- 
ating teaching can yield the most useful information for in5)lementing 
state educational leadership. At the most general Iwel, there are 
three such conditions which apply: the infomation must he relevant, 

interpretahle, and reliable. 

Providing for Relevance, Intei^retability, 
and Reliability 

Relevance refers to the extent of the relationship existing 
between the information collected and the circumstances which state 
educational leadership is intended to affect. In general, this leader- 
ship is directed toward helping local education authorities to estab- 
lish .educational prograins which meet the needs of their citizenry and 
toward atteu^ting to bring about conditions whereby the objectives 
of these programs are.* most likely to be attained. Thus, the actual 
relevance of infomation on teaching is best detemined by the person- 
nel in the local schools, except in areas in which the state specifies 
certain teaching processes to be followed or certain teaching products 
to be sought. However, states generally do not impose such specification 
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even though they may he endorsed hy special interest groups. The 
usual position of educational leaders on this -matter has been the 
one represented in the guidelines set forth in* (Chapter I of the 
present study; namely, that the state, when exercising leadership, 
should neither pre-empt local initiative in determining educational 
objectives, nor dictate to professional educators the prpcedures which 
they should select to accomplish these objectives. This is in line 
with- the concept of state educational leadership presented by 

1 

Thurston and Roe in their book on State School Administration . They 
feel that state departnients of education can be more effective if they 
place their greatest emphasis upon techniques other than regulation. 

To provide for the maximum relevance in terms of types of 
performance and types of results expected from teachers within the 
schools of the state, it is proposed that a statewide program for 
evaluating teaching use as its base data collected through evaluation 
programs designed to meet local needs. If these programs are carefully 
developed in terns of local educational goals, the resulting data are 
likely to be more relevant than data obtained from any other source. 

■ In striving for relevance in educational research, the problem 
of rigor, which is a prerequisite for interpretability, is often en- 
countered. Such is the case in the present situation. Attempting to 
combine data collected in different places under different conditions 
is certain to bring about problems in interpretation. These are 

^Lee M. Thurston and William H. Roe, "The Growing Edge," in 
their St ate School Administration (New York: Harper and Brothers, 

19 57 ), Tp. 40^419^ 
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discussed below. Also pertinent are remarks later in the chapter 
relating to pilot programs for developing exemplary evaluation methods 
and methods for disseminating information. 

While the proposed approach admittedly contains inherent dif- 
ficulties, the drawbacks of the alternatives seem even more critical. 
The alternatives would be either to require local school officials to 
collect information specifically for the state program, or for the 
state to anploy special personnel to go into the schools and collect 
the infoimation. With either approach, the nature of the information 
collected would still be determined by local educaticnal goals and 
policies, if the guidelines stated in CHiapter I are not to be violated. 
However, with these approaches, the initiative for evaluating teaching 
would be taken from the local personnel and assumed by the state. 

This is contrary to the philosophy of state educational leadership 
which has been endorsed in the present study. If the information 
were collected for the state by local school personnel, it seems 
likely that the evaluation mi^t become a perfunctory administrative 
task. (There have been informal reports that the present Florida 
teacher evaluation program has been treated in some schools as Just 
such a chore.) Because of the inherent difficulties in evaluating 
teaching, such an 'approach is apt to provide data which lack reliabil- 
ity. If, on the other hand, a staff of state evaluators were employed, 
it might appear to many that education had taken a step backward with 
t?ie return of the state school inspector. The reception accorded such 
an official would certainly not be favorable. 
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Thus, to reiterate, the most reasonable approach for obtaining 
relevant data on ■which to base a. program for utilizing information 
on t-eaching to implement state educational leadership is to make use 
of data collected through carefully developed local evaluation programs. 
This practice would assure that data obtained were relevant to the 
purposes toward which the local educational programs are directed. 

Using data from local evaluation programs would not, however, 
preclude the possibility of conducting analyses which do not relate 
directly to the criteria adopted in individual schools. It would be 
possible to conduct pilot tests of numerous hypotheses by extracting, 
transforming, combining, or otherwise modifying the data collected. 

One important type of extra analysis would relate to the performance 
of graduates from different kinds of teacher education programs. The 
data could also be used in conjunction with data collected frcm other 
sources. Some of the types of analyses which could be conducted are 
described by Medley and Mitzel in their discussion of possibilities 
for mailing maximum use of data collected in natural (as opposed to 
experimental) s ituations . ^ 

If a state program for measui’ing and evaluating teaching relies 
on data collected in individually developed local programs, difficul- 
ties are almost certain to develop when it comes time to combine 
data from different schools. This problem can be classed in the realm 

^Donald M. Medley and Harold E. Mitzel, "Measuring Classroom 
Behavior by Systematic Observation," Handbook of Research on Teaching, 
ed. N. L. Gage (Chicago: Rand McNally and Company, I 963 ), pp. 321- 

325. 




of interpretability, the second condition necessary in a state program 
to provide information on teaching. Inteipretability refers to con- 
ditions which allow the data to be organized and analyzed in ways 



which will yield information that can be used for desired pi.irposes, 
in this ' fi.-.r i/ho 'onplementation of state educational. Loa/iership. 

It is thus apparent that relevance is a necessary, but not a suf- 



ficient, condition for inte] 7 pretability. As previously stated, the 
information collected from 3.ocal school districts would be relevant 
as it would represent the conditions which, in the judgment of the 
professional personnel of the individual schools, are basic to accomplish- 
ing the goals of the individual school programs. The presence of rele- 
vance would not guarantee, however, that the measuring scales and obser- 
vation methods used in different schools would be designed in a manner 
which would permit data from different evaluation programs to be 
combined. 

On the other hand, the idea of imposing conditions which would 
allow for the combining of data from, unique locally developed programs 
for evaluating teaching seems antithetical to the position developed 
\mder the discussion of relevance which encourages the exercise of 
local prerogative. Nevertheless, it seems unlikely that every pro- 
gram developed will be canpletely unique. There will certainly be 
common elements running through many of them. Once local school dis- 
tricts have developed their criteria, it would be possible for the 
state to exercise leadership in surveying, analyzing, and comparing 
thes<'.' criteria. The common elements could be identified. The state 
could encourage communication between schools which use similar 
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approaches and possibly coiild support ot sponsor workshops for training 
observers in which representatives of several schools WDuld participate. 
Thus^ the state would not be discouraging local initiative, but wou3.d 
be helping schools to do a better Job of evaluating teaching, en^loy- 
ing the criteria which they deemed significant. 

As evaluation programs develop and gain some stability, it 
might be possible for a taxonomy of criteria for evaluating teaching 
to be developed, comprising the core or most basic elements which are 
included in the various programs for evaluating teaching found through- 
out the state. With such a taxonomy available, it might be practical 
to devote considerable effort and expense to developing and validating 
instruments for measinring the factors which relate to the classified 
criteria. Thus, a common library of professionally developed and 
standardized instruments would be available for use by those schools 
which found them appropriate. In using these instruments it VDuld 

I 

still be possible for different schools to select different scales 
and assign different weights to those selected. Their use WDuld con- 
tribute significantly to interpretab ility. In addition, the possi- 
bility for local schools to use only those scales which they deemed 
relevant and to apply unique weights to those selected provides for 
local initiative. 

The most essential prerequisite for interpretability is suf- 
ficient knowledge on the part of the person who is analyzing the data 
of the conditions under which they were collected. Thus, a further 
step for enhancing interpretability would be to file information on 
these conditions with the data record. The conditions could include 
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not only the time, place, and circumstances of the observation, but 
also relevant facts about the training of the observer. 

The third condition necessary, if infomation on teaching is 
to be of value in inrplementing state educational leadership, is reli- 
ability. This term refers to the consistency between information on 
teaching which is collected and the behaviors or conditions which that 
information is assumed to represent. In other words, if precisely the 
same, type of teacher behavior were to take place in two different 
schools and this behavior were recorded by observers in both ochools, 
it should be possible to ascertain from the independently prepared 
observation records that the behavior in the two schools was identical. 
When the rating scales or other measures used provide a stable and 
veridical representation of the teaching to which they pertain, the 
resulting data are high in reliability. 

Since reliability of measures is essential in local evaluation, 
whether or not the data are to be used elsewhere, the use of data 
co]lecbe.fl in carefully developed local programs should ensure some 
measure of re.l lability. To increase this likelihood, local schools 
cind. school districts could be provided consultation and other assist- 
ance in developing scales and in training observers. It should be 
recognized, however, that such procedures will contribute to reliabil- 
ity only for measures taken wa.thin schools. They will not assure 
t}iat; measures taken with the same scale in two different school dis- 
tricts will be at all consistent. Further efforts would be needed such 
as programs which woiQ.d bring together people from different schools 



o 

ERIC 






■ 8l 

for the purpose of discussion and practice in obseiring teaching 
processes and products. 

Obviously, the problems of inter-school reliability and 
interpretability of data collected from various sources are closely 
intertwined. Therefore, the proposals made in the preceding discussion 
of interpretability are relevant here also. The development of a 
taxonomy of teaching criteria and of carefully constructed instruments 
for measuring the various elements of the criteria would certainly con- 
tribute to the reliability of data collected in more than one school 
or school system. 

Much of the above discussion relates most directly to rating 
or other types of systema'^ic observation of teacher or pupil perform- 
ance. If tests are used to evaluate teaching in terms of product 
criteria, procedures for . obtaining reliability and interpretability 
which have already been treated extensively in the literature on 
testing are iimnediately applicable. 

Collecting and Storing the Data 

To conduct the desired analyses, it is necessary to have in- 
formation on teaching which is collected throughout the state assembled 
in such a manner that it will be accessible when it is needed. Because 
of the volume of the information and the complex problems inherent in 
selecting the information for any given analysis, the proposed system 
is designed with electronic data processing equipment in mind. The 
data which are collected would be stored on magnetic tape, magnetic 
discs, or in some other form allowing for access via computer. 
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The data collected from loca"*' school districts would include 
not only the information which represents measures of the teaching 
which has taken place^ but also information which establishes its 
relevance and provides guidance in its i.\terjoretation. Since the 
system is intended to be used to provide infomation for use in mak- 
ing decisions relative to teacher education, infomation obtained 
from teacher education institutions pertaining to appropriate pro- 
cedures and criteria for evaluating their graduates would also be 
obtained. 

The data would be stored in a number of separate data files. 
Appropriate infomation could be selected from any one of these files 
or any combination of these files for analyses which might be con- 
ducted in reference to numerous specific questions. Descriptions of 
the proposed data files are given below. 

Infomation on Criteria 

The first two files would contain statements of criteria and 
procedures which are deemed appropriate for evaluating teaching in 
terms of the criteria. Infomation for these files would be obtained 
from all school districts in the state and frcm all programs for pre- 
paring teachers within the state. 

Teacher education program evaluation criteria . —This file will 
be a composite record of the criteria which each of the teacher educa- 
tion institutions considers appropriate for evaluating tne teaching 
of its graduates. A set of criteria could be applicable to all 
graduates of an institution, to graduates of a specific program within 
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an institution, or only to individual teachers. The criteria would 
relate to the performance of teachers and/or to the outcomes of their 
teaching. Each set of criteria would he stated in a manner sufficiently 
specific for determining whether or not it would be appropriate to use 
them in conjunction with teaching performance o: teaching product infor- 
mation obtained from the various local school districts where teachers 
to whom the criteria are applicable might be teaching. 

School program evaluation criteria .— This file will contain 
a compilation of the criteria which local school systems consider 
relevant for evaluating teaching in their schools along with the pro- 
cedures which are to be employed for evaluation. As with the criteria 
in the preceding file, a given set could apply to all teachers in an 
entire district, to teachers of a given subject, at a given level, 
in a specific school, teaching a specific type of class, or to an 
individual teacher only. The criteria would relate to the performance 
of teachers and/or to the outcomes of their teaching. 

Status Information 

•The second group of files would contain information on the 
status of persons certified to teach in the state and on the schools 
in the state. This information wou3.d be ccmpiled from certification 
and accreditation records. Information in these files would pertain 
to all teachers and to all schools. 

School status inf ormation . —This file will include information 
on schools which is considered significant in evaluating the quality 
of the school program. It would relate to school organizations. 
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cuxxicul'cuii^ snd. sJLl o'fcli©!' fs-cbors "wiiicb. sr© giv6n offlclsX. 
consideration when determining the eligibility of a school for 
accreditation. 

Teacher status infor mation . - -This file will contain informa- 
tion on individual teachers which is considered relevant in predicttog 
teaching coii 5 )etence. This would Include those factors which are used 
to determine whether or not a baccalaureate graduate should be recom- 
mended for a teaching certificate, and whether or not such a certifi- 
cate should be issued. 

Evidence of Teaching 

The third set of files would contain information on teaching 
performance and teaching products which have been observed and re- 
corded. This information will be collected by local school personnel 
while carrying out local programs for evaluating teaching. It is 
not anticipated that all evaluation data collected by local schools 
would be forwarded to the state department of education for inclusion 
in these files. The state personnel would analyze information in 
the criteria files and then request from local school districts the 
information which would be .isable for analysis in reference to 

specific questions of interest. 

Teaching performance information . --This file will contain 
measures of the teaching performance of individual teachers. The 
measures on each teacher will be obtained with instruments and pro- 
cedures which provide for interpretability, employing the criteria 
specified for that teacher in the file of school program evaluation 




criteria. 
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Pupil performance information * "-This file will contain measures 
of the performance of pupils tau^t by teachers who are to be evaluated 
employing this type of criterion. The measures will, of course, be 
in a form which will allow them to be related to the applicable 
criteria. 



Analyzing the Data 

The general procedure proposed ;for lanalyzing the data involves 
two phases: the comp;ilation of the file of data for a specific analysis 

and the analysis, itself. The general, design for the analysis system 
is depicted in Figure 8. 

The compilation of the analysis file will consist of selecting 

I 

from the data files the information which will be analyzed. This will 
involve selecting records from individual files according to given 
selection criteria, and/or combining, records or items from, individual 
files to build new records . 

The number of specific an^yses. which mi^t b^ conducted employ- 
ing the data in the files described: is virtually unlimited. Some 
analyses which relate to the specific questions listed in .Chapter I 
are considered below. A restatement of. the applicable question or 
questions precedes each discussion. 

(l) What criteria do individual colleges think are relevant for 
evaluating the teaching of graduates of their programs? Are 
these the same for all graduates? For all graduates in given 
teaching fields? 

The information for answering this question would be contained in the 
file of teacher education program evaluation criteria. In order to 
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DATA FILES 



Criteria Status Performance 





Fig. 8— General Design for the Data Analysis System. 
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answer the question posed, it would be necessary to develop a system 
for classifying the criteria, possibly according to the dimensions of 
teaching to which they relate. Examples of different dimensions 
stated in terms of teacher behavior might include those used by re- 
searchers: e.g., verbal influence of the teacher as analyzed by 
Flanders, logical use of inguages as analyzed by Smith, or phil- 
osophic concepts related to teaching performance as described by 
Brown’. Or, they might include more traditional categories such as 
maintaining discipline, using teaching aids, or providing for individ\ial 
differences. Dimensions stated in terns of pupil performance might be 
described in terms of the categories in the Taxonomy of Educational 
Objectives : e.g., knowledge, comprehension, application, responding, 

valuing. As soon as the criteria are classified, the above question 
can be answered by tabulating the classification information. The 
answers can be provided in several forms: the information could be 

tabulated by subject specialty, by institution, by type of institution, 
by geogra; lic area, or by w’hatever unit of analysis is called for by 
the specific problem at hand. 

(2) How does the teaching done by graduates of different types of 
teacher education programs measure up against the evaluation 
criteria considered appropriate by the teacher education in- 
stitutions? By the local school districts? 

The analyses relevant to this question would begin by identifying the 

teachers whose teaching is to be studied. This could be done with the 

file of teacher status information which would contain information on 

the programs of preparation conq)leted by individual teachers. The 

criteria that have been established for evaluating these particular 
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teachers would then be reviewed using both the teacher education pro- 
gram criteria file and the school program criteria file. Finally, 
usable information on those teachers from the file of teaching per- 
formance information and/or the file of pupil performance information 
would be analyzed. 

(3) What criteria do individual school districts or schools judge 
to be relevant for evaluating teaching in their classrooms? 

Are these the same for all or most teachers within a school 
district? Within a school? For all or most teachers of 
certain grades or subjects? 

This question is almost the same as the first question, the difference 
being that it pertains to criteria established by local school dis- 
tricts while the earlier one referred to criteria established by 
teacher education institutions. The procedure would be basically the 
same as described earlier:: (l) to identify the schools to be included 

in the population for analysis, ( 2 ) to classify the criteria, and 
(3) to tabulate the data so that they can be comprehended. 

(il-) How does the teaching done by teachers in individual schools 
measure up against the evaluative criteria which are set 
forth as appropriate by the school district? 

The first step in dealing with this question would be to determine 

which schools were to be selected and how the teachers from those schools 

were to be grouped. It would be necessary also to determine what phases 

of teaching were of interest. The necessary records from either the 

pupil performance or. the teaching performance files, depending upon 

the type of criteria employed in the schools selected, could be used 

and the data could be tabulated or otherwise organized into a foim which 

would provide for inteipretabllity. 
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(5) Ho¥ do the criteria specified for different segments of the 
teaching population (e.g., beginning teachers, music teachers, 
sixth grade teachers, etc.) compare? 

This question would require that the segments of the teaching popula- 
tion to he analyzed first he identified, with the use of the teacher 
status file, and that the criteria established for evaluating the 
work of these people then he selected from the teacher education pro- 
gram criteria f:Lle and/or the school program criteria file. The 
analyses would consist of describing the data through tabulation or 
other tecliniqu.es. 

(6) How do evaluation criteria selected by school districts and 
evaluation criteria selected by colleges compare? 

The first step in dealing with this question would be to select a 
population of teachers, .possibly on the basis of their having com- 
pleted selected teacher education programs. The teacher status file 
could be used for this purpose. Records for these teachers on the 
school program criteria file and the teacher education program criteria 
file could then be compared. The results of the comparison could be 
summarized. 

(T) Is there more than one distinctive pattern of expectaticns for 
teachers in a given field? 

The analyses for this question employ data from the two criteria files. 
The criteria which are applicable to teachers in. the specific field 
under consideration Wbuld be selected and tabulated. This is the 
same type of analysis proposed in connection with the questions deal- 
ing with types of criteria specified by teacher education Institutions 
or local school systems. 
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( 8 ) What are the inservice education needs of various segments 
of the teaching population? 

Information relevant to this question would be obtained by securing 
from the files of teacher education program evaluation criteria and 
school program evaluation criteria an indication of the dimensions 
of teaching which are considered relatively important for a group to 
which a program of inservice education would be directed. Such a 
group might be composed of teachers of a given subject or teachers in 
a given geographic area. Information would then be retrieved from 
the teacher performance file which would indicate the relative strengths 
and weaknesses of teachers in each of the significant aspects of teaching. 

(9) What are the relationships between teacher status characteristics 
( viz .^ the things which are considered in recommending a 
baccalaureate candidate for a teaching certificate or in issu- 
ing a teaching certificate) and teaching (evaluated in terms of 
criteria established by locaJ. school districts)? 

The first step in dealing with this question would be to determine which 

specific status factors and \diich specific dimensions of teaching are 

of interest. Then, records containing measures of the significant 

dimensions of teaching can be selected from the file of teaching 

I « 

performance data. The desired status data on the teachers represented 
can be selected from the file of teacher status information and the 
two sets of information can be compared. 

, (10) What relationships exist between school status characteristics 
( viz ., the things which are given consideration in recommend-,.? 
ing a school for accreditation) and the teaching and learn- 
ing which takes place in those schools. 

This question would require the same basic procedures as the preceding 

one. It would first be necessary to identify the specific status 
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■ factors and also the specific types of pupil or teacher performance 
■which are of interest. After selecting the records containing the 
information to he analyaed, the data could he compared. 

Initial Steps in Developin g a State Program to 
Collect and Analyze Information on Teaching 

Implementation of the system for collecting and analyzing in- 
formation on teaching which is descri}^^' ihi .^Jhis chapter involves 
three phases: (l) developing in local schools or school districts 

carefully conceived programs for evaluating teaching, (2) developing 
communication and collaboration between schools and school districts 
which will make possible the combining of data from different evalua- 
tion programs, and (3) developing a central system for storing and 
analyzing the data collected. In the design of the total system, 
these phases are interrelated; therefore, it is proposed that they 
be pursued concurrently, rather than sequentially. Various ideas 
which warrant consideration in carrying out these phases are discussed 
below. 

Inducing Local School Districts to 
Establish Evaluation Programs 

Local programs for the evaluation of teaching are neither new 
nor uncommon. Surveys of teacher rating practices in public schools 
date back at least to 1915.^ The most recent National Education 

^Hazel Davis, "Evolution of Current Practices in Evaluating 
Teacher Competence," Contemporary Research on Teacher Effectiveness , 
edr Bruce J. Biddle and William J. Ellena (New York: Ho3.t, Rinehart 

and Winston, 196k), pp« 4l-66. 
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'Association survey of this type reports that a ma^jority of large 
school systems and at least a fourth of the smaller school systems 
represented in its nationwide sauries have formal progr'ams for evalu- 
ating teaching.^ Thus, the problem" of getting local evaluation pro- 
grams started is reduced considerably » 

An obvious way to induce local school districts which do not 
have programs for the evaluation of teaching to establish them would 
be to pass a law or a regulation. However, such an action would not 
be likely to promote an attitude on the part of local ochool personnel 
which would result in the type of commitment necessary to develop high 
quality programs. Development of the desired evaluation programs 
would require conscientious local initiative in seeking a definition 
as to what, under specified circumstances, constitutes good teaching. 
Once this is defined, there is the equally challenging problem of 
developing objective procedures for collecting and analyzing the 
necessary evidence to deteimine whether good teaching has taken place. 

A tendency could develop to perform tasks of th;’ ' ^ype in a superficial 
manner if their sole purpose were to fulfill the requirements of a 

statute . 

It might be advisable, however, to establish an official regu- 
lation or policy affirming the desirability of locally developed 
evaluation programs. Efforts could also be made to support pilot pro- 
grams which could demonstrate the application of various approaches. 

^Hazel Davis, Evaluation of Classroom Teachers, Research Re- 
port 1964-R14 (Washington, D. C.: Research Division, National Educa- 

tion Association, 1964), p. l4. 
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Eventually, vhen most school districts have developed sound programs, 
it ‘might he advisable to pass a lav or regulation to pull along those 
vho .had not done so, if it vere felt that all schools should have a 
formal program for evaluating teaching. 

In cases vhere statutes are in effect vhlch specify the forms 
and procedures to he used for evaluating teaching, the evaluation lavs 
vould, of course, have to he repealed or amended to allov local school 
districts to develop their ovn evaluation programs. While conditions 
vary from state to state, an amendment vould prohahly he preferable 
to repealing the lav completely. This is suggested to avoid convey- 
ing the iu5)ression that the state no longer thinks it inrportant to 
evaluate teaching. The amended lav vould need to provide only (l) 
that all teachers he evaluated annually, at least, en5>loying criteria 
developed by the professional personnel vlthin the district, and 
(2) that, upon request, criteria and procedures employed in each 
district and data collected in the evaluation program he submitted 
to the state department of education. The first stipulation vould 
offer assurance that the state is still interested in protecting its 
citizenry from incompetent teaching. The second provision vould 
officially give the state access to the data needed for the informa- 
tion system, hut vould not require schools to furnish the state 
department of education vith information vhich it vas not prepared 
to put to use. 
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Pilot Pro.jects to Develop Exemplary 
Programs for Evaluating Teaching 

The conceptuaJLization of evaluation criteria and the trans- 
lation of those criteria into operational terms as required by the 
proposed evaluation system are demanding tasks. To increase the likeli- 
hood of developing programs of exceptional quality, it is proposed that 
a number of pilot projects be instituted which would make use of 
resources beyond those that can normally be committed in a single 
school or district. For such projects, supplemental financing could be 
made available to allow certain school district personnel to devote a 
greater proportion of their time to the evaluation project and also 
to bring in consultative assistance from outside the district. Con- 
sultants might include scholeirs in educational philosophy, authorities 
on the principles and techniques of teaching, and experts in research 
and measurement. Pilot projects could be initiated by colleges and 
universities, by school districts, or jointly by colleges or univer- 
sities aLid school districts. 

Each of the projects would identify the dimensions of teaching 
to be incorporated in its criteria, define the criteria, develop and 
validate the necessary measurement instruments, and demonstrate the 
application of the evaluation program in an ongoing school setting. 
Listinmients could be demonstrated and tested in the pilot school and 
in other schools which volunteered to participate. 

The ultimate aim of the pilot projects would not be to impose 
upon all schools certain criteria or procedures for evaluating teach- 
ing. Nor would the projects be intended in any manner to restrict 
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other schools in the development of their own criteria and measuring 
devices. The goal would he siu^ly to elevate the standards of evalua- 
tion in given schools and to make available information regarding 
procedures employed. 

Disseminating Information on Newly Developed 
Programs for the Evaluation of Teaching 

Although it is not proposed that methods for measuring and 
evaluating teaching developed in pilot projects be imposed upon any 
school, it is assumed that if the methods have merit they will be 
ador ‘:ed. This is, of course, contingent upon the prospective adopters 
being introduced to the newly developed evaluation systems. A number 
of procedures for communicating this information might usefully be 
en5)loyed. These include publications; conferences or workshops at 
the state, regional, county and local school levels; and informal 
consultation. 

The publications could include magazine articles or brochures 
describing the various approaches to the evaluation of teaching bein^ 
taken in different pilot projects and their stages of development. 

When an evaluation system has been developed and demonstrated, detailed 
bulletins could be published describing the rationale employed in the 
development of the evaluation system and the specific procedures for 
applying it. Data could also be given regarding the reliability and 
validity of the measuring techniques employed. 

Conferences or workshops of a variety of types could be 
scheduled for a number of different purposes. Conferences could be 
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scheduled in advance of the pilot projects to arouse interest in partici- 
pation. As the pilot projects progress, conferences could he held to 
demonstrate, ccmpare, and evaluate the approaches which are being 
taken in the varioiis projects. This may also point out the need for 
additional projects. When an evaluation program reaches a stage call- 
ing for testing of the procedures at other schools, conferences could 
be held to demonstrate the applicability of those procedures. When a 
project I..as validated its measuring procedures, conferences could be 
held to demonstrate them to persons who might be interested in using 
them. When a school has chosen to adopt a particular evaluation 
system, a workshop can be scheduled to instruct teachers and adminis- 
trators in its use. 

Virtually all of the activities described in the preceding 
paragraphs could also be accomplished through informal meetings 
and consultation. The particular approach selected would depend upon 
the specific cir.cumstances in each situation. 

Developing the Central Data System 

The planning of the central data system should begin as soon 
as the decision is made to develop a state system for organizing and 
analyzing information on teaching. There should be continual ccm- 
rndnication between the persons designing the data system and those in 
local school systems to make possible optimum adaptability of the data 
system to the anticipated data and, when feasible, to provide for 
optimum adaptability of the anticipated data to the data system. This 
is not to say that evaluation programs should be designed for the 
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convenience of the data processors; if, however, certain modifications 
of evaluation procedures which would increase data processing capabili- 
ties could be accomplished without affecting the underlying rationale 
or the overall quality of the evaluation program, they should certainly 
be considered. It is particularly important that communication be 
maintained between schools carrying out pilot projects and the designers 
of the central information system since it is assumed that procedures 
from the successful projects will be adopted in numerous additional- 
schools. 

The actual building of the files niust wait until data are 
available, although fictitious data could be used to try out processing 
programs and procedures. Data for the status files would be available 
Immediately, however, since this information is nomally contained in 
state department of education records. Information for criteria 
files would be available before information for perfomance files, as 
the data for the latter are dictated by the criteria. 
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RECAPITULATION AND REFLECTIONS 

This study was instituted to delineate the conditions under 
which statewide programs for measuring and evaluating teaching can 
yield optimum information for use by the states in carrying out their 
educational leadership responsibilities. The responsibilities under 
consideration relate particularly to three state services administered 
through state departments of education: the education of teachers 

the certification of teachers, and the accreditation of schools. 

Recapitulation 

Chapter I discusses the types of activities carried out by 
state governments in their efforts to provide for high quality teach- 
ing in elementary and secondary schools. These include (l) develop- 
ing and administering procedures to insure the quality of programs 
for the preparation of teachers^ (2) developing and administering 
standards which will allow only qualified people to hold teaching 
positions, and (3) developing and administering programs for promot- 
ing teacher growth and elhninating substandard teaching situations. 
State educational leadership consists of devising ways to improve 
these activities. Comprehensive information on the teaching which is 
taking place in the schools would be of great value in Implementing 
this leadership. 
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It is noted that virtually no systematic use of such informa- 
tion is presently made by state governments. This could be attributable 
to the fact that much information on teaching which has been or might 
be collected is ambiguous. There is no widespread agreement as to 
what criteria sould be used for evaluating teaching. Therefore, con- 
fusion may result when two people analyze the same evidence because 
they are using different criteria. Much research has been conducted 
to determine the factors which can account for effectiveness of in- 
effectiveness in teaching. This has generally not been fruitful. A 
further complicating factor relates to the tenuous relationship be- 
tween teaching and the subsequent behavior of pupils. The influence 
of the teacher is only one of the many effects which contribute to 
pupil learning. 

In Chapter II, the three statewide programs for evaluating 
teaching which are presently operative are described. The State of 
Pennsylvania has prescribed a form and procedures for rating teachers. 
Satisfactory ratings on. the form are required in order for a teacher 
to be eligible for tenure and in order for him to have his certificate 
renewed. An unsatisfactory rating, along with supporting evidence, 
provides a legal basis for dismissing a tenure teacher on grounds of 
incompetence. The* Pennsylvania program is not designed to provide 
information which can be used in making decisions relative to teacher 
education, certification and accreditation. 

The statewide evaluation program in Hawaii has as its stated 
purpose the improvement of teaching in the schools of that State. It 
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is a program similar to one which might he developed in a large school 
district. The state prescribes a form and procedures; principals 
observe the teachers and, following the visits, review the ratings 
in a conference with the teachers. The program for evaluating the 
work of tenure teachers performs no official regulatory function. 
However, as in Pennsylvania, satisfactory ratings are required for a 
probationary teacher to become eligible for tenure. The Hawaii pro- 
gram is not designed to provide information which can be used to re- 
view the effectiveness of state educational policies and programs. 

In Florida, all teachers are evaluated annually using a form 
prescribed by the State. The results are placed on file in the State 
Department of Education in a manner which allows them to be summarized 
and analyzed using electronic data processing equipment. While no 
official purposes have been specified for this program, it is organized 
in a manner which should provide the kind of information which could 
be used t^ maj^ . dec is ions regarding state policies for improving 
teaching. However, an examination of data collected indicates various 
shortcomings which restrict considerably interpretations that can be 
made using this information. 

Chapter III discusses approaches to the evaluation of teaching. 
The process of evaluation, as described in the present study, consists 
of comparing evidence with criteria. Consequently, the nature of the 
evidence which is relevant is determined by the criteria. Three types 
of criteria— product, process, and presage— are described. Either 
the product type or the process type can be used in evaluating; presage 
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criteria are, in reality, predictors. The selection of criteria con- 
stitutes a value judgment which may be quite arbitrary. However, a 
person who is familiar with the situation to which the criteria will 
apply is normally in a better position to make a judgment as to what 
constitute acceptable criteria. 

The process of collecting evidence and ccmparing it with 
criteria is viewed along three dimensions. They relate to (l) the 
nature of the situation in which the evidence is collected (varying 
from "normal” to "constructed"), (2) the agent responsible for col- 
lecting the evidence (with the performer," himself, represented at one 
extreme and observer at the other), and (3) the point at which the 
evidence is compared with the criteria (varying from "while observ- 
ing" to "independently"). 

The applicability of various techniques for collecting evidence 
and comparing it with criteria are discussed. These include self 
reports and self evaluation, rating and rating scales, systematic 
observation with evaluation performed independently, and testing. 

There are conditions under which each of these techniques might be 
applicable in a statewide program for obtaining infomation to 
implement educational, leadership. 

Chapter IV describes an approach for organizing and analyzing 
infomation on teaching to implement state educational leadership. 

The effectiveness of such a program depends first upon three con- 
ditions which must be met by the infomation used; they are, relevance, 
intei^retability, and reliability. Relevance pertains to the 
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relationship between the information collected and the circumstances 
which state educational leadership is intended to effect. Because 
this leadership is aimed primarily at helping local school districts 
to carry out programs which meet the needs of their citizens, it is 
proposed that the most relevant information can be obtained from 
carefully designed local evaluation programs. This, however, could, 
lead to problems of interpretability. 

Inteipretability is the second essential condition for an 
effective program, infoimation which is interpretable can be summarized 
and analyzed in ways which will yield information that can be used for 
the desired purposes, in this case, for the inqplementation of state 
educational leadership. Atten^ting to combine data collected in dif- 
ferent places under different circumstances is certain to bring about 
problems of interpretability. To deal with these, it is proposed that 
communication be developed between schools using similar criteria, 
that pilot programs for the development of exemplary evaluation 
systems be established and the results widely disseminated, and that 
a taxonomy of criteria including the core or most basic elements 
which are included in evaluation programs throughout the state be 
compiled. The taxonomy could be the starting point for the develop- 
ment and standardization of instruments for measirring various elements 
,of teaching. 

Reliability, the third essential condition for an effective 
program refers to the consistency between the information on teaching 
which is collected and traits, behaviors, or conditions which that 
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information is assumed to represent. Reliability results when care- 
fully developed instruments and procedures are used. 

Six different files are proposed for collecting and storing 
the information in the state department of education. They would 
contain information on (l) teacher education program criteria, (2) 
school program criteria, (S) teacher status info 2 ma.tion, (4) school 
status information, (5) teaching performance information, and ( 6 ) 
pupil performance information. These files would be maintained in a 

form which would allow access via computer. 

The general procedure for analyzing the data involves two 
phases. The first consists of selecting from the files the data for 
analysis. The data could come from any one file or any combinat?_on 
of files. The selected data would be incorporated into a new file. 

The second phase consists of the analysis, itself. This would utilize 
descriptive or in ferential statistical technic^ues. Brief illustra- 
tions of applications for the analysis system are presented in 
Chapter IV . 



Reflections 

If the specifics of the proposals made in this study are pushed 
into the background, the general message can be summarized in the form 
of two proposals,. First, state governments should broaden their com- 
mitment to institutional research as a basis for decision-making in 
the area of education. And second, educators in the elementary and 
secondary schools should begin to codify their professional wisdom. 
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Codifying the Wisdom of Educational Praxititioners 

The proposal for codifying the knowledge of practitioners 
developed in the present study as an outgrowth of the institutional 
research idea. It resulted from the concept that evaluation must be 
based upon criteria and the stipulation that neither the teaching 
processes to be employed nor the teaching products to be sought should 
be dictated by the state to local school personnel. This means that, 
if evaluation is to take place, local school personnel must establish 
the criteria. The establishment of criteria by such personnel would 
constitute a codification of their professional wisdom. 

The resulting statements would set forth the factors which, 
in the judgment of practicing educators, determine sound educational 
practice. It would not be a superficial document as might result 
from a simple survey or Q-sort. Since the statement of criteria would 
form the official basis for evaluating the work of teachers, it would 
be reasoned out in a manner which would strike for the essence of the 
issues. As the criteria were applied, the codification would be con- 
tinually re-evaluated, revised, and improved. As communication and 
collaboration between schools increased, the body of knowledge develop- 
ed by professional practitioners would become more refined. 

Lortie^ declares that a codified body of knowledge representing 
the best thinking of skilled professional practi oners is something 

. ^Dan C. Lortie, "Teacher Socialization: The Robinson Crusoe 

Model," The Real World of the Beginning Teacher (Washington, D. C.: 
National Commission on Teacher Education and Professional Standards, 

1966), pp. 5^-66. 
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which is lacking in the field of education. This is not the case, how- 
ever, in the professions of medicine, law, and architecture. He notes 
that, while the professional subject matter for the teacher contains 
contributions from philosophers and psychologists, it does not in- 
clude a body of knowledge codified by eduu_obional practitioners. 

In medicine, law, and architecture there are such courses, based upon 

1 

the cumulated wisdom of practitioners. 

Lortie also observed that there exists a chasm between schools 
of education and teanhers in the elementary and secondary schools. 

He asserts that teachers do not attribute' the same in^ortance to 

2 

professional schooling as do physicians, lawyers, and architects. 
Research results documenting the gap between education professors and 
teachers are reported by Joyce, ^ who found that attitudes toward 
teaching held by beginning teachers are more in agreement with those 
held by the general public than with those held by profe ’.sors of 
education. 

Tiis codifying by practitioners of a body of practical knowledge 
could contribute to the improvement of teacher education and also to 
the narrowing of the breach between practitioners and professors. 

^Lortie gives Internal Medicine, Torts, and Principles of 
Design as exaio^les of this type of covrse. Ibid ., p. 60.. 

2 ^ 

Ib-id . , p. d2. 

^Bruce R. Joyce, "The Social Climate of Teacher Education," 
Journal of Teacher Education, Vol. ik (jiine, 1963 )^ PP» 179-183*- 



o 



lo6 



This knowledge would give beginning teachers invaluable support as 
they commence their professional duties. Excerpts from this body of 
knowledge would certainly be adopted by schools or departments of 
education and integrated into their programs. Hopefully, the ultimate 
result would be the wedding of the best elements from both the pro- 
fessors and the practitioners. One effect of this would be a more 
prominent place for elementary and secondary classrooms in programs 
of teacher education. 

It is often stated that professional education for teachers 
should become more closely associated with the classroom. Typical 
recommendations are that longer student teaching periods be estab- 
lished, that internship programs be developed, that students be 
assigned case studies or other projects which will bring them into 
the schools, or that instructo..'s in graduate courses slant their 
presentations more directly toward the problems of the teachers 
enrolled. 

It is seldom suggested that the wisest and most skillfiiL 
teachers perform the difficult task of codifying their professional 
wisdom so that it can benefit both them and their associates. Yet, 
it seems unnecessary that each first-year teacher should have to 
begin anew to develop such a body of professional wisdom. It also 
seems unreasonable that a professor of education should have to 
relate his psychology or philosophy to chance exar^les when a body of 
professional information codified by educational practitioners might 
be a^'^ailable to form a basis for discussion. The codification of the 
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knowledge of practicing educators could serve to bring teacher prepar- 
ation and teaching closer together. It would provide for an easier 
induction of the beginning teacher and a basis for communication be- 
tween teachers and education professors from which they could proceed 
to work together to improve teaching. 

Broadening the State Commitment 
to Institutional Research 

The present study began by articulating a case for institutional 
research^ without using the term. Institutional research is intended 
to provide data which makes it possible for administrators or leaders 
to make intelligent decisions. It represents a rational, rather than 
arbitrary, approach to decision-making. It is widely practiced in 
certain phases of state school administration. On matters relating 
to educational finance, such as cost of certain programs or tax- 
paying ability in certain areas or of certain segments of the popula- 
tion, institutional research is the rule, rather than the exception. 
Institutional research is also practiced extensively in anticipating 
needs for new facilities. However, as was pointed out in Chapter I, 
very little use is made of institutional research in developing 
policies for teacher education, certification, and accreditation. 

The primary proposal, in this study is that attempts be made 
to take a rational approach to these matters, as is typically taken 
in the domains of finance and facilities. Optimum decisions are more 
likel:-" to result when as much relevant information as possible can 
be brought to bear. 
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Some Side Effects 

There axe at least two areas not yet mentioned in which imple- 
mentation of the above general prpposals would contribute to the im- 
provement of teaching. These areas represent fimctions which are, in 
fact, more fundamental in iu^Jroving teaching than are the state services 
toward which uhe infoimation system is directed. The first relates to 
research on teaching; the second to stimulation of local leadership 
for the improvement of instruction. 

In the area of research, the large volume of carefully collected 

measures and evaluations of teaching would make data available for 

numerous correlation studies or other investigations en 5 )loying pre- 

1 

experimental or quasi-experimental designs. These would include 
studies probing relationships between various statiis characteristics 
of teachers and various types of performance of teachers or between 
various types of performance of teachers and various teaching 
products . 

It is also expected that implementation of the system would 
lead to improved understanding and respect by school personnel of the 
researcher's approach. This would result from continual efforts by 
teachers and administrators to define their criteria and to enploy 
measures for determining the extent to which they have been met. This 

^Donald T. Campbell and Julian C. Stanley, "Experimental and 
Quasi-Experimental Designs for Research on Teaching," Handbook of 
Research on Teaching, ed. N. L. Gage (Chicago: Rand McNally and 

Company, 1963 PP* 171-246. 
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is the same basic problem ■which the research "worker encounters re- 
peatedly throughout his career. 

The understanding which is expected to result should make the 
schools more accessible for use as laboratories for experimental studies. 
Moreover, it seems likely that the greater sophistication of the 
school people would contribute to better experimental controls and 
better research results. 

Instructional leadership at the local level would also be 
affected by the greater "understanding of research. The search for 
criteria and for methods of applying them could make school personnel 
more alert and receptive to research results which might provide 
assistance or guidance. 

However, the greatest stimulus to instructional improvement 
within the local schools would almost certainly result from the 
initiative taken by local school personnel to define and redefine 
the criteria which they wish to employ in evaluabing teaching. With 
definite criteria at hand and definite evidence as to the extent to 
which the criteria are being met, more decisive measures can be 
taken to improve teaching. 

Conclusion 

This study has proposed the development of an information 
system which is capable of providing facts which can be used by the 
state in strengthening educational leadership. The system is de- 
signed to allow the state to make decisions relative to policies for 
teacher education, teacher certification, and school accreditation 




on a more rational "basis. The implementation of the system, however, 
would provide some additional "benefits which are, in reality, more 
significant than state regulations and policies for improving teaching. 
These include (l) increasing the potential contri"butions of prac- 
titioners to teacher education, (2) stimulating educational research 
and the application of research findings, and (3) stimulating local 
leadership for the improvement of instroiction. 
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DIVISION OF TFACHF.R EDUCATION, CERTIFICATION AND ACCREDITATION 
FLORIDA STATE DEPARIMENT OF EDUCATION 
TALUHASSEE 32304 



StaUwlde Te acher Evaluation Queatlonnalre 



Haa your state within the last ten yeara conducted a atatawlda 
teacher evaluation project? 



y*«. 



no 



Do you conduct regular annual statavlda taachar 

evaluattona? V**. 



Indicate the yeara of your three laoat recent atatcwlde 
teacher e'aluatlon projecta* 

1966 .1965 1964 1963 1962 1961 1960 1959 1958 1937 



Answer the following queatlona for each of your three moat recant atatawlda 
taacher evaluation projecta. 



moat recent 


earlier 


earlieat of the 




project 


project 


three projecta 


Waa it conducted in an effort 





y«« 


y«« 


to recognize and promote 
merltorloua teaching? 


no 


no 


no 


Waa it conducted in an effort 


y«« 


ye« 


y«« 


to recognize and eliminate 
Incompetent teaching? 


no 


no _ 


no 


Va? It conducted for the 


y«« 


y«« 


y«« 


purpoae of accumulating 
data for research? 


no 


no 


no ■ 


Was It conducted for purposes 


yes 


ye« 


y«« 


othe.r than those »wiitloned nbove? 


no 


no 


no 



PLEASE ENC1.0SE POLICY STATEMENTS, EVALUATION INSTRUMENTS, PROJECT REPORTS, OR 
ANY OTHER AVAIURLE MATERIAL WHICH IS RELEVANT TO YOUR STATEWIDE TEACHER 
EVALUATION PROJECTS. 



(person completing questionnaire) 



(title) 
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FLOYO T. CHRItTIAN 
•UrtmNYKNDKNT 



State of PijOrida 

Department of Education 

TAUAHASStC 31304 

August 1, 1966 



We would like to request your assistance In obtaining Information on 
statewide teacher evaluation practices In yiiur state. This Information 
Is solicited as part of a larger study of practices and potential for 
statewide teacher evaluation programs. 

Statewide teacher evaluation Is defined here as an attempt by the state 
government (or Its agent) to make a systematic composite assessment of 
the quality o.f teaching or teachers within the entire state. 

Your cooperation Is greatly appreciated. 

Sincerely yours, 



Wm. Cecil Golden, Director 
Division of Teacher Education, 
Certification and Accreditation 

WCG’.ad 



Enclosures: 



Questionnaire 
Return Envelope 





rtOVO T. CHRISTIAN 
tUrCRi.N'IXNDbNT 



State op Florida 

Department op Education 

TAllAHASSEE 33304 

August 29, 1966 



About four weeks ago we requested Information from you relative to 
statewide teacher evaluation practices. Since I have not yet received 
your reply, I am enclosing a second copy of the questionnaire In the 
event that the first might have been mislaid. 

I hope that you will return the questionnaire. Your assistance Is 
greatly appreciated. 

Sincerely, 



Wm. Cecil Golden, Director 
Division of Teacher Education, 
Certification and Accreditation 



WCG:ad 

Enclosures: Questionnaire 

Return Envelope 
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The Multi-State Teacher Education Project 



The Multi-State Teacher Education Project (M-STEP) was initiated to 
strengthen the contributions of state departments of education to teacher 
education. This is being done through innovative pilot projects being 
carried out in each of the seven M-STEP states. The M-STEP design was 
adopted to enhance cooperation between the participating states in plan- 
ning, analyzing, and evaluating the pilot projects and also in dissemi- 
nating information on activities and results. In Florida, pilot projects 
intended to help teacher education institutions and elementary and sec- 
ondary schools to obtain information which they can use in improving pre- 
sc-rvice and inservice teacher' education are now underway. These projects 
have taken two forms: (1) bringing people together in the State to 

exchange ideas, and (2) developing techniques for analyzing data col- 
lected by the State Department of Education to provide usable information. 
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